-- Logs begin at Wed 2022-08-24 09:13:58 UTC, end at Wed 2022-08-24 10:41:06 UTC. -- Aug 24 09:56:10.349969 dv-cinder-854271-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 09:56:10.359352 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 09:56:10.361384 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 09:56:10 2022] *** Aug 24 09:56:10.361384 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 09:56:10.361384 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 09:56:10.361384 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: nodename: dv-cinder-854271-2-iscsi Aug 24 09:56:10.361384 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: machine: x86_64 Aug 24 09:56:10.361384 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: clock source: unix Aug 24 09:56:10.361384 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: pcre jit disabled Aug 24 09:56:10.361384 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: detected number of CPU cores: 4 Aug 24 09:56:10.361384 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: current working directory: / Aug 24 09:56:10.361384 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: detected binary path: /usr/bin/uwsgi-core Aug 24 09:56:10.361384 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: your processes number limit is 47680 Aug 24 09:56:10.361986 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: your memory page size is 4096 bytes Aug 24 09:56:10.361986 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: detected max file descriptor number: 2048 Aug 24 09:56:10.361986 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: lock engine: pthread robust mutexes Aug 24 09:56:10.361986 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: thunder lock: enabled Aug 24 09:56:10.361986 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 09:56:10.361986 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 09:56:10.401534 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: Python main interpreter initialized at 0x559beb0799c0 Aug 24 09:56:10.401534 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: python threads support enabled Aug 24 09:56:10.401534 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: your server socket listen backlog is limited to 100 connections Aug 24 09:56:10.401534 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: your mercy for graceful operations on workers is 90 seconds Aug 24 09:56:10.401913 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 09:56:10.401992 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: *** Operational MODE: preforking *** Aug 24 09:56:10.402231 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: *** uWSGI is running in multiple interpreter mode *** Aug 24 09:56:10.402231 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: spawned uWSGI master process (pid: 81919) Aug 24 09:56:10.402441 dv-cinder-854271-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 09:56:10.403303 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: spawned uWSGI worker 1 (pid: 81930, cores: 1) Aug 24 09:56:10.403303 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: spawned uWSGI worker 2 (pid: 81931, cores: 1) Aug 24 09:56:10.403459 dv-cinder-854271-2-iscsi devstack@keystone.service[81919]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 09:56:14.753833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:56:14.753833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: from cryptography.utils import int_from_bytes Aug 24 09:56:14.754641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:56:14.754641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: from cryptography.utils import int_from_bytes Aug 24 09:56:14.756213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:56:14.756213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: from cryptography.utils import int_from_bytes Aug 24 09:56:14.756681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:56:14.756681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: from cryptography.utils import int_from_bytes Aug 24 09:56:14.805185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 09:56:14.805377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 09:56:14.805550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] command line args: [] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 09:56:14.805814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 09:56:14.806061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ================================================================================ {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 09:56:14.806900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 09:56:14.807034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] admin_token = **** {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.807143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] config_dir = [] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.807229 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 09:56:14.807336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.807516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] config_source = [] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.807599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] command line args: [] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 09:56:14.807693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] debug = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.807818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 09:56:14.807901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.808190 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ================================================================================ {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 09:56:14.808298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.808470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] admin_token = **** {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.808580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.808677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] config_dir = [] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.808769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.808908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.809055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.809161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] config_source = [] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.809264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] list_limit = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.809364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] debug = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.809467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] log_config_append = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.809561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.809692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.809692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] log_dir = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.809867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.810024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] log_file = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.810137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.810264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.810378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.810460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.810570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.810693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.810790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] list_limit = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.810872 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.810997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] log_config_append = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.811079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.811203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.811287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.811406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] log_dir = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.811406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] log_file = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.811590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.811671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.811792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.811878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.811951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.812181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.812270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.812270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.812398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.812538 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.812653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.812653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.812843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.812937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.813070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.813152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.813275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.813351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.813479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.813562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.813689 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.813802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] publish_errors = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.813911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.813998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.814124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.814204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.814299 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.814398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.814544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.814678 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.814767 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.814997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] standard_threads = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.815183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.815262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.815369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] publish_errors = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.815467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.815603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.815681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.815762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] use_journal = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.815839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.815936 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] use_json = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.816013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.816110 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] use_stderr = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.816234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.816319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] use_syslog = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.816470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.816548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] standard_threads = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.816805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.816985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.817193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.817278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.817351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.817427 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.817506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.817578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] use_journal = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.817703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.817783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] use_json = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.817931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] use_stderr = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.818010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.818115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] use_syslog = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.818199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.818351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:56:14.818436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.818609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.818694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.818851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.818934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.819096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.819174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.819328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.819415 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.819573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.819657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.819805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.819884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.820050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.820167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.820286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.820400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.820649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.820723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.820829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.821023 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.821112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.821270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.821362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.821506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.821603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.821815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.821944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.822071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.822160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.822324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.822403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.822576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.822654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.822810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.822885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.823020 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.823098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.823309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.823391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.823559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.823639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.823800 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.823878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.824045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.824121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.824275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.824353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.824504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.824581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.824749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.824827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.825019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.825106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.825256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.825340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.825503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.825590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.825722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.825805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.825955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.826040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.826188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.826298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.826407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.826504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.826787 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.826906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.827040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.827146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.834952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.835107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.835233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.835350 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.835482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.835599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.835729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.835854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.835965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.835965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.835965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.836287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.836410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.836519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.836648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.836756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.836891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.837017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.837144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.837270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.837381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.837493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.837603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.837728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.837837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.837967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.838101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.838215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.838349 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.838458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.838581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.838704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.838816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.838928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.839039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.839159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.839277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.839400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.839521 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.839630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.839740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.839852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.839972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.840092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.840201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.840322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.840434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.840557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.840666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.840790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.840938 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.841057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.841169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.841293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.841404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.841530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.841639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.841751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.841862 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.841999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.842131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.842348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.842468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.842468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.842673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.842784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.842895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.843018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.843128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.843248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.843358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.843469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.843582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.843724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.843724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.843956 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.844066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.844177 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.844287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.844399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.844527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.844646 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.844755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.844887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.845008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.845122 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.845282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.845446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.845446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.845655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.845767 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.845943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.846087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.846244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.846351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.846470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.846585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.846703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.846836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.846947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.847069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.847185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.847309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.847423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.847560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.847672 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.847788 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.847904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.848019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.848135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.848265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.848386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.848499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.848615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.848736 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.848972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.849104 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.849220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.849340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.849466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] auth.external = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.849466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.849685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.849685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.849913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.850033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.850151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.850270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.850394 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] auth.password = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.850528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.850642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] auth.token = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.850762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.850889 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.851006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.851125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.851254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.851369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.851528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.851648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.851766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.851887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.851887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.852105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.852241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.852355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.852484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] credential.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.852611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] auth.external = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.852730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.853353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.853487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.853600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.853729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.853857 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.853968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.854091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] auth.password = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.854220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.854340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] auth.token = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.854456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.854572 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.854687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.854803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.854926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.855141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.855258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.855376 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.855493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.855610 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.855739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.855851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.855963 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.856095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.856209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.856338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] credential.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.856468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.856579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.856695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.856822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.856979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] federation.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.857097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.857305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.857416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.857537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.857663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.857783 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.857904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.858036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.858152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.858273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.858395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.858515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.858636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.858769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.858886 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.859006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.859006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.859231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.859409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.859520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.859644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.859764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.859890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.860010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.860131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.860247 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.860359 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.860469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] federation.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.860596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.860717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.860850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.861036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.861155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.861271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.861388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.861505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.861640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.861640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.861859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.861974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.862090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.862215 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.862345 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.862345 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.862558 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.862682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.862808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.862922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.863050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.863164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.863281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.863396 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.863514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.863629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.863757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.863871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.863987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.864103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.864103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.864331 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.864464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.864586 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.864701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.864816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.864816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.865055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.865188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.865301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.865418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.865536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.865653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.865770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.865904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.866031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.866151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.866273 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.866394 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.866513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.866644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.866762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.866885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.867006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.867126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.867246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.867379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.867496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.867624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.867745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.867865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.867986 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.868119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.868239 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.868359 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.868359 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.869262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.869393 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.869522 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.869642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.869763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.869888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.870006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.870139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.870256 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.870376 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.870494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.870613 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.870734 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.870865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.870981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.871101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.871220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.871339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.871460 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.871595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.871709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.871830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.871830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.872053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.872170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.872310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.872435 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.872552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.872672 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.872793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.872928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.873068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.873184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.873301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.873418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.873534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.873649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.873779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.873891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.874007 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.874124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.874241 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.874357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.874485 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.874599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.874715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.874836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.874953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.875070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.875199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.875319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.875430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.875538 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.875648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.875760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.875884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.875995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.876498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.876618 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.876736 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.876877 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.877025 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.877140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.877261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.877383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.877502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.877625 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.877759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.877875 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.877997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.878118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.878239 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.878359 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.878502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.878616 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.878738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.878857 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.878978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.879107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.879236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.879355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.879476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.879595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.879715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.879836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.879969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.880085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.880085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.880304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.880430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.880546 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.880678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.880795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.880983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.881093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.881213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.881329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.881531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.881642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.881760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.881876 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.881999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] resource.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.882113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.882245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.882356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.882473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.882599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.882712 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.882829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.882958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.883072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.883181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.883181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.883382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.883493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.883614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.883744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.883862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.883972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.884096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.884220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] role.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.884350 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.884465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] role.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.884575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.884703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.884821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.884981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.885106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.885229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.885347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.885465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.885584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.885714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.885829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.885945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.885945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.885945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.886249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.886378 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.886496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] resource.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.886616 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.886737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.886866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.886984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.887116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.887235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.887354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.887354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.887569 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.887569 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.887791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.887905 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.888026 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.888147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.888267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.888392 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.888530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.888648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.888768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.888974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] role.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.889085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.889204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] role.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.889339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.889450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.889569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.889685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.889801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.889921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.890056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.890168 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.890285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.890402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.890517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.890634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.890762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.890875 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.890991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.891106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.891222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.891337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.891465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.891577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.891698 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.891814 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.891930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.892058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.892171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.892288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.892404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] token.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.892404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.892622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.892749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.892749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.892983 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.893098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.893220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.893336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.893472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.893584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.893701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.893816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.893816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.894036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.894163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.894163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.894380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.894495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.894612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.894729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.894863 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.894975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.895093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.895210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.895338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.895451 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.895580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.895690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.895807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.895921 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.896048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.896162 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.connection = **** {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.896293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.896429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.896542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.896654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.896759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.896878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.896989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.906944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907104 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907346 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] token.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907513 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907736 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907880 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.907960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908253 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.908954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909027 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81930) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 09:56:14.909341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.connection = **** {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.909964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.910504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.910504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.910504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.910504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.910504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.910504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.910504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:56:14.910504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81931) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 09:56:14.932040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:14.932266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:14.932502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:14.933129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:14.933367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:14.935772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:14.941034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:14.942346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:14.969296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:14.969522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:14.971044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:14.971274 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.030838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.031318 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.083960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.084189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.084282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.084403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.247071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.247311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.247531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.247756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.247976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.248218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.248679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.248937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.249161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.249379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.249600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.249813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.272531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.272736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.272736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.272928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.273008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.273084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.273201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.273288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.273422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.273508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.273636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.273730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.273851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.273937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.285941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.286354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.299567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.299795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.300019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.300218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.300306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.300446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.300530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.300658 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.300743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.300884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81930) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.300970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.301123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.301341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.301559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81931) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:56:15.317618 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.317831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:56:15.318169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.318364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:56:15.318451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.318636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.318729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:56:15.318896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.318980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.319182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:56:15.319414 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.319494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.319677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.319766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.320223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.320489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.329143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.329421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.329690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.329970 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.330050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.330272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.330550 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.330840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.336883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.338408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.340250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.340539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.340819 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.341130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.341346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.341429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.341652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.341733 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.341919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.342003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.342188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.342270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.342485 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.342565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.342749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.342831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.343022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.343101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.343298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.343692 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.343975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.344251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.365791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.367246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.367611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.369711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.369998 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.372832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.373133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.373417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.374679 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.374964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.375251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.379635 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.379918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.381597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.381887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.384048 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.384989 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.385272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.385552 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.385822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.386093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.386362 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.386530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.386626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.386825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.387113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.387399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.387677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.387955 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.388229 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.388503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.403806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.404089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.406143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.406435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.407984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.410345 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.411207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.411502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.413580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.413890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.415833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.418436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.419226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.421618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.422460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.422758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.423044 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.423329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.423605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.423895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.424931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.425235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.425522 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.425808 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.426092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.426396 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.440136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.440423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.440688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.440978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.442737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.443058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.443336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.443627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.448497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.450031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.450314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.451202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.452738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.452936 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.453027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.453223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.453508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.453792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.454063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.455707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.455998 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.456292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.456581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.456890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.465418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.465706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.465986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.466265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.466545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.466822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.467145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.468235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.468530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.468841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.469153 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.469460 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.469764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.470077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.482501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.485498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.485766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.488337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.488799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.489798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.491269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.491549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.491703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.491838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.493188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.495540 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.495845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.496151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.498903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.500348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.502632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.503255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.503550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.503841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.504097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.504192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.507090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.507380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.507681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.507960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.512706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.513016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.513298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.516892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.517198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.517485 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.520061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81930) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.520348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.520628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.520999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.521272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.521550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.521821 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.522098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.522375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.522650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.522930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.523190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.523463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.523729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81930) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.524422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81931) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:56:15.524778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.525112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.525476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.525781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.526059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.526337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.526631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.526910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.527190 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.527476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.527749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.528018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.528302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81931) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:56:15.559456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559beb0799c0 pid: 81930 (default app) Aug 24 09:56:15.562916 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa37ad-0bc1-49a9-888c-9c69eef04690 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:15.563200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa37ad-0bc1-49a9-888c-9c69eef04690 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:15.563470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa37ad-0bc1-49a9-888c-9c69eef04690 None None] PATH_INFO: `/v3` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:15.564362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559beb0799c0 pid: 81931 (default app) Aug 24 09:56:15.567591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Wed Aug 24 09:56:15 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:23.617630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf992311-8ab6-4bbe-94fc-4a8c6ae8ae30 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:23.618182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf992311-8ab6-4bbe-94fc-4a8c6ae8ae30 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:23.618182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf992311-8ab6-4bbe-94fc-4a8c6ae8ae30 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:23.622856 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:56:23.629857 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ab04b-787c-4c32-b337-f4aa8b066885 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:23.630283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ab04b-787c-4c32-b337-f4aa8b066885 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:23.630403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ab04b-787c-4c32-b337-f4aa8b066885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:23.703312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: INFO dbcounter [None req-a96ab04b-787c-4c32-b337-f4aa8b066885 None None] Registered counter for database keystone Aug 24 09:56:23.724984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG oslo_db.sqlalchemy.engines [None req-a96ab04b-787c-4c32-b337-f4aa8b066885 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81930) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:56:23.732256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writer thread running {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:56:23.918733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG passlib.handlers.bcrypt [None req-a96ab04b-787c-4c32-b337-f4aa8b066885 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81930) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 09:56:23.919068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG passlib.handlers.bcrypt [None req-a96ab04b-787c-4c32-b337-f4aa8b066885 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81930) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 09:56:23.975263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-a96ab04b-787c-4c32-b337-f4aa8b066885 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:23.976917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a96ab04b-787c-4c32-b337-f4aa8b066885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:24.110203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 09:56:24.612483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf992311-8ab6-4bbe-94fc-4a8c6ae8ae30 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:24.683512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: INFO dbcounter [None req-bf992311-8ab6-4bbe-94fc-4a8c6ae8ae30 None None] Registered counter for database keystone Aug 24 09:56:24.702840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG oslo_db.sqlalchemy.engines [None req-bf992311-8ab6-4bbe-94fc-4a8c6ae8ae30 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81931) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:56:24.709049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writer thread running {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:56:24.829413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf992311-8ab6-4bbe-94fc-4a8c6ae8ae30 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:24.833646 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6fc25-b008-4cdf-bd81-9ad4f0e4eb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:24.834275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6fc25-b008-4cdf-bd81-9ad4f0e4eb36 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:24.834553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6fc25-b008-4cdf-bd81-9ad4f0e4eb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:24.834824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6fc25-b008-4cdf-bd81-9ad4f0e4eb36 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:24.843216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6fc25-b008-4cdf-bd81-9ad4f0e4eb36 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:24.895751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: WARNING keystone.common.rbac_enforcer.enforcer [None req-67f6fc25-b008-4cdf-bd81-9ad4f0e4eb36 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 09:56:24.896015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6fc25-b008-4cdf-bd81-9ad4f0e4eb36 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:24.906662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Wed Aug 24 09:56:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:56:24.912631 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ab04b-787c-4c32-b337-f4aa8b066885 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:24.927978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ab04b-787c-4c32-b337-f4aa8b066885 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:24.932563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b7436-3a18-4e69-83b7-5b8ff5550459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:24.933712 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b7436-3a18-4e69-83b7-5b8ff5550459 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:24.933995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b7436-3a18-4e69-83b7-5b8ff5550459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:24.934268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b7436-3a18-4e69-83b7-5b8ff5550459 admin admin] PATH_INFO: `/v3/projects/85765b8a176a42c8b2b1e092bbcf9c4f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:24.935829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b7436-3a18-4e69-83b7-5b8ff5550459 admin admin] RBAC: Authorizing `identity:get_project(project_id=85765b8a176a42c8b2b1e092bbcf9c4f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:24.989462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ed5b7436-3a18-4e69-83b7-5b8ff5550459 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 09:56:24.989826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b7436-3a18-4e69-83b7-5b8ff5550459 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:24.994364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Aug 24 09:56:24 2022] GET /identity/v3/projects/85765b8a176a42c8b2b1e092bbcf9c4f => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:26.916122 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b5c2e-164e-4bc9-8ab4-4b60aee4210d None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:26.916951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b5c2e-164e-4bc9-8ab4-4b60aee4210d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:26.917207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b5c2e-164e-4bc9-8ab4-4b60aee4210d None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:26.921712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:26.928973 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe13d-e345-472d-a390-6876e0031f14 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:26.929474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe13d-e345-472d-a390-6876e0031f14 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:26.929567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe13d-e345-472d-a390-6876e0031f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:26.950371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e7fbe13d-e345-472d-a390-6876e0031f14 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:26.951910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e7fbe13d-e345-472d-a390-6876e0031f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:26.959836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 09:56:27.231474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b5c2e-164e-4bc9-8ab4-4b60aee4210d None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:27.243685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b5c2e-164e-4bc9-8ab4-4b60aee4210d None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:27.249251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a36f91-d999-4a02-b20b-9b87f2bb0452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:27.249878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a36f91-d999-4a02-b20b-9b87f2bb0452 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:27.250162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a36f91-d999-4a02-b20b-9b87f2bb0452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:27.255909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a36f91-d999-4a02-b20b-9b87f2bb0452 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:27.258999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a36f91-d999-4a02-b20b-9b87f2bb0452 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:27.261694 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a36f91-d999-4a02-b20b-9b87f2bb0452 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:27.271921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Wed Aug 24 09:56:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:56:27.277837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbe13d-e345-472d-a390-6876e0031f14 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:27.288662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbe13d-e345-472d-a390-6876e0031f14 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:27.294820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5c9df-8072-4c30-b672-6d132f863210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:27.295457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5c9df-8072-4c30-b672-6d132f863210 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:27.295821 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5c9df-8072-4c30-b672-6d132f863210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:27.296107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5c9df-8072-4c30-b672-6d132f863210 admin admin] PATH_INFO: `/v3/users/deb4aa9488e54d0eafc940ddd57f345d` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:27.297472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5c9df-8072-4c30-b672-6d132f863210 admin admin] RBAC: Authorizing `identity:get_user(user_id=deb4aa9488e54d0eafc940ddd57f345d)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:27.299942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5c9df-8072-4c30-b672-6d132f863210 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:27.302249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Wed Aug 24 09:56:27 2022] GET /identity/v3/users/deb4aa9488e54d0eafc940ddd57f345d => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:29.346757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446060f-5a14-45d7-a1b7-414ba4cb9225 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:29.347333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446060f-5a14-45d7-a1b7-414ba4cb9225 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:29.347333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446060f-5a14-45d7-a1b7-414ba4cb9225 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:29.348698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:29.355210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7cb6f-44ae-4a64-ae29-9544b1302c2b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:29.355848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7cb6f-44ae-4a64-ae29-9544b1302c2b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:29.355848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7cb6f-44ae-4a64-ae29-9544b1302c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:29.371244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253765bd-c2f2-490a-8fe9-ca9838e2177a None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:29.371530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253765bd-c2f2-490a-8fe9-ca9838e2177a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:29.371813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253765bd-c2f2-490a-8fe9-ca9838e2177a None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:29.373243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:56:29.376466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-eed7cb6f-44ae-4a64-ae29-9544b1302c2b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:29.377700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-eed7cb6f-44ae-4a64-ae29-9544b1302c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:29.379252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7284065-19b1-489e-b4cb-3473ac46ca43 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:29.379596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7284065-19b1-489e-b4cb-3473ac46ca43 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:29.379965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7284065-19b1-489e-b4cb-3473ac46ca43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:29.426954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG passlib.handlers.bcrypt [None req-b7284065-19b1-489e-b4cb-3473ac46ca43 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81931) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 09:56:29.427667 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG passlib.handlers.bcrypt [None req-b7284065-19b1-489e-b4cb-3473ac46ca43 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81931) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 09:56:29.472035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:56:29.475613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-b7284065-19b1-489e-b4cb-3473ac46ca43 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:29.477417 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b7284065-19b1-489e-b4cb-3473ac46ca43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:29.595377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:56:29.744751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7cb6f-44ae-4a64-ae29-9544b1302c2b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:29.769501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:29.866330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7284065-19b1-489e-b4cb-3473ac46ca43 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:29.868306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7cb6f-44ae-4a64-ae29-9544b1302c2b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:29.903305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc3fdd-ce69-4249-9b6a-df5be52efca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:29.903942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc3fdd-ce69-4249-9b6a-df5be52efca9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:29.904277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc3fdd-ce69-4249-9b6a-df5be52efca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:29.904494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc3fdd-ce69-4249-9b6a-df5be52efca9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:29.917703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc3fdd-ce69-4249-9b6a-df5be52efca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-08fc3fdd-ce69-4249-9b6a-df5be52efca9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:29.926201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:29.927129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:56:29.928258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:56:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:29.933659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc3fdd-ce69-4249-9b6a-df5be52efca9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:29.978912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7284065-19b1-489e-b4cb-3473ac46ca43 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:29.981470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:30.016869 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31941a-ba91-445c-b309-6688b579a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:30.017776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31941a-ba91-445c-b309-6688b579a699 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:30.018056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31941a-ba91-445c-b309-6688b579a699 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:30.018332 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31941a-ba91-445c-b309-6688b579a699 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:30.019085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:30.037811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc3fdd-ce69-4249-9b6a-df5be52efca9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:30.072928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74fa32-8964-4e7e-b276-51a64cd37f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:30.073542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74fa32-8964-4e7e-b276-51a64cd37f4d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:30.073823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74fa32-8964-4e7e-b276-51a64cd37f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:30.074095 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74fa32-8964-4e7e-b276-51a64cd37f4d None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:30.074758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74fa32-8964-4e7e-b276-51a64cd37f4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:30.075056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74fa32-8964-4e7e-b276-51a64cd37f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:30.076442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74fa32-8964-4e7e-b276-51a64cd37f4d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:30.088592 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 8/15] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:30.088841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31941a-ba91-445c-b309-6688b579a699 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:30.090394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31941a-ba91-445c-b309-6688b579a699 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:30.094200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74fa32-8964-4e7e-b276-51a64cd37f4d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:30.190811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 8/16] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:56:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:56:30.194867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74fa32-8964-4e7e-b276-51a64cd37f4d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:30.196323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31941a-ba91-445c-b309-6688b579a699 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:30.200054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:30.231146 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a4ff-e2ef-4e2a-b103-7d26ad57efa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:30.231782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a4ff-e2ef-4e2a-b103-7d26ad57efa5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:30.232064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a4ff-e2ef-4e2a-b103-7d26ad57efa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:30.232338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a4ff-e2ef-4e2a-b103-7d26ad57efa5 None admin] PATH_INFO: `/v3/users/deb4aa9488e54d0eafc940ddd57f345d` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:30.233555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63a4ff-e2ef-4e2a-b103-7d26ad57efa5 None admin] RBAC: Authorizing `identity:get_user(user_id=deb4aa9488e54d0eafc940ddd57f345d)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:30.235010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63a4ff-e2ef-4e2a-b103-7d26ad57efa5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:30.236589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 9/17] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:30 2022] GET /identity/v3/users/deb4aa9488e54d0eafc940ddd57f345d => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:30.242324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a4ff-e2ef-4e2a-b103-7d26ad57efa5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:30.258401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:30.302649 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31941a-ba91-445c-b309-6688b579a699 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:30.339444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7559ddc-a33f-4d90-be94-04df8ea5b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:30.340293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7559ddc-a33f-4d90-be94-04df8ea5b56f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:30.340570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7559ddc-a33f-4d90-be94-04df8ea5b56f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:30.340838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7559ddc-a33f-4d90-be94-04df8ea5b56f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:30.350367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a4ff-e2ef-4e2a-b103-7d26ad57efa5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:30.359784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7559ddc-a33f-4d90-be94-04df8ea5b56f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:30.361301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7559ddc-a33f-4d90-be94-04df8ea5b56f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-a7559ddc-a33f-4d90-be94-04df8ea5b56f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:56:30.380596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:30.381563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:56:30.382412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:56:30.382412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:56:30.382412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:56:30.382412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 9/18] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:56:30.388419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7559ddc-a33f-4d90-be94-04df8ea5b56f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:30.389879 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0d356-e3e0-4e3f-a1b0-a3cd7997a748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:30.390535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0d356-e3e0-4e3f-a1b0-a3cd7997a748 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:30.390856 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0d356-e3e0-4e3f-a1b0-a3cd7997a748 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:30.391142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0d356-e3e0-4e3f-a1b0-a3cd7997a748 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:30.392322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0d356-e3e0-4e3f-a1b0-a3cd7997a748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:30.393865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0d356-e3e0-4e3f-a1b0-a3cd7997a748 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:30.395422 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 10/19] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:56:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:30.401286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0d356-e3e0-4e3f-a1b0-a3cd7997a748 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:30.445515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:30.485328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:30.491721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7559ddc-a33f-4d90-be94-04df8ea5b56f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:30.504140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0d356-e3e0-4e3f-a1b0-a3cd7997a748 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:30.526968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b6650-35d5-4796-b4f6-f838b2cd710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:30.527585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b6650-35d5-4796-b4f6-f838b2cd710a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:30.527868 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b6650-35d5-4796-b4f6-f838b2cd710a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:30.528143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b6650-35d5-4796-b4f6-f838b2cd710a None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:30.528785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b6650-35d5-4796-b4f6-f838b2cd710a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:30.529195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b6650-35d5-4796-b4f6-f838b2cd710a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:30.530589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b6650-35d5-4796-b4f6-f838b2cd710a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:30.540010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ab45e-c38a-41b2-a6a7-98632e21ccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:30.540674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ab45e-c38a-41b2-a6a7-98632e21ccee None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:30.541021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ab45e-c38a-41b2-a6a7-98632e21ccee None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:30.541267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ab45e-c38a-41b2-a6a7-98632e21ccee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:30.542243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ab45e-c38a-41b2-a6a7-98632e21ccee None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.domain.id=default, user.id=deb4aa9488e54d0eafc940ddd57f345d) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:30.542539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ab45e-c38a-41b2-a6a7-98632e21ccee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:30.543962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ab45e-c38a-41b2-a6a7-98632e21ccee None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:30.550274 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 10/20] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:30.550845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 11/21] 10.222.0.30 () {56 vars in 1322 bytes} [Wed Aug 24 09:56:30 2022] GET /identity/v3/role_assignments?user.id=deb4aa9488e54d0eafc940ddd57f345d&scope.domain.id=default&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:32.781469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edff22-1293-47fa-8122-3caed8214537 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:32.782029 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edff22-1293-47fa-8122-3caed8214537 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:32.782029 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edff22-1293-47fa-8122-3caed8214537 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:32.783365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 11/22] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:32.789215 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d732-b3ac-4af0-bb5e-f42a3cdb610e None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:32.789671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d732-b3ac-4af0-bb5e-f42a3cdb610e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:32.789763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d732-b3ac-4af0-bb5e-f42a3cdb610e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:32.818401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-30b5d732-b3ac-4af0-bb5e-f42a3cdb610e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:32.819596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-30b5d732-b3ac-4af0-bb5e-f42a3cdb610e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:32.919507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 12/23] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:33.065339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc7b14-e91f-45b3-a05e-a354347bdbce None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:33.065644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc7b14-e91f-45b3-a05e-a354347bdbce None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:33.065825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc7b14-e91f-45b3-a05e-a354347bdbce None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:33.067423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 12/24] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:33.078757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc9dc-4932-4130-8f47-68f872edcd91 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:33.079078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc9dc-4932-4130-8f47-68f872edcd91 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:33.079320 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc9dc-4932-4130-8f47-68f872edcd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:33.105919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-8a3fc9dc-4932-4130-8f47-68f872edcd91 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:33.107140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8a3fc9dc-4932-4130-8f47-68f872edcd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:33.112986 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd93eb-7088-4c34-ab83-86bbe16d5e30 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:33.113272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd93eb-7088-4c34-ab83-86bbe16d5e30 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:33.113545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd93eb-7088-4c34-ab83-86bbe16d5e30 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:33.115168 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 13/25] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:33.127040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4b6b-c761-4179-a5e1-9c0310a854e8 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:33.127325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4b6b-c761-4179-a5e1-9c0310a854e8 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:33.127604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4b6b-c761-4179-a5e1-9c0310a854e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:33.166750 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-03af4b6b-c761-4179-a5e1-9c0310a854e8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:33.168034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-03af4b6b-c761-4179-a5e1-9c0310a854e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:33.243321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 13/26] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:33.322446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 14/27] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:56:33.338749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fc9dc-4932-4130-8f47-68f872edcd91 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:33.353791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:33.448458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fc9dc-4932-4130-8f47-68f872edcd91 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:33.485803 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562f931-eaeb-48f5-8f61-4c5786be4a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:33.486448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f931-eaeb-48f5-8f61-4c5786be4a6b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:33.486731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f931-eaeb-48f5-8f61-4c5786be4a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:33.487054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f931-eaeb-48f5-8f61-4c5786be4a6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:33.492514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7ccf2-d04f-481a-b647-8cc8891ca8a9 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:33.492800 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7ccf2-d04f-481a-b647-8cc8891ca8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:33.493128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7ccf2-d04f-481a-b647-8cc8891ca8a9 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:33.494642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 15/28] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:56:33.498648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562f931-eaeb-48f5-8f61-4c5786be4a6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:33.502097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25169f25-30af-4a9e-87c5-0300f0e95e1d None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:33.502385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25169f25-30af-4a9e-87c5-0300f0e95e1d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:33.502666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25169f25-30af-4a9e-87c5-0300f0e95e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-8562f931-eaeb-48f5-8f61-4c5786be4a6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:33.504052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:33.504907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:56:33.505750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 14/29] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:56:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:33.511703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562f931-eaeb-48f5-8f61-4c5786be4a6b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:33.552674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-25169f25-30af-4a9e-87c5-0300f0e95e1d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:33.553990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-25169f25-30af-4a9e-87c5-0300f0e95e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:33.609747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:33.623806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562f931-eaeb-48f5-8f61-4c5786be4a6b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:33.635597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:33.661799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058cb09-7c0c-481a-9b65-02b1100f4e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:33.662526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cb09-7c0c-481a-9b65-02b1100f4e47 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:33.662944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cb09-7c0c-481a-9b65-02b1100f4e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:33.663691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cb09-7c0c-481a-9b65-02b1100f4e47 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:33.666119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058cb09-7c0c-481a-9b65-02b1100f4e47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:33.668467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058cb09-7c0c-481a-9b65-02b1100f4e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:33.669120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058cb09-7c0c-481a-9b65-02b1100f4e47 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:33.674019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 16/30] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 09:56:33.675208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25169f25-30af-4a9e-87c5-0300f0e95e1d None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:33.680762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 15/31] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:33.687530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058cb09-7c0c-481a-9b65-02b1100f4e47 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:33.855518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058cb09-7c0c-481a-9b65-02b1100f4e47 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:33.880186 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25169f25-30af-4a9e-87c5-0300f0e95e1d None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:33.890592 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:33.899740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308149b3-95cc-445b-bedc-e2b38f537e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:33.906867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308149b3-95cc-445b-bedc-e2b38f537e9e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:33.906867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308149b3-95cc-445b-bedc-e2b38f537e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:33.906867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308149b3-95cc-445b-bedc-e2b38f537e9e None admin] PATH_INFO: `/v3/users/deb4aa9488e54d0eafc940ddd57f345d` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:33.906867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308149b3-95cc-445b-bedc-e2b38f537e9e None admin] RBAC: Authorizing `identity:get_user(user_id=deb4aa9488e54d0eafc940ddd57f345d)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:33.906867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308149b3-95cc-445b-bedc-e2b38f537e9e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:33.906867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 16/32] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:33 2022] GET /identity/v3/users/deb4aa9488e54d0eafc940ddd57f345d => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:33.911532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308149b3-95cc-445b-bedc-e2b38f537e9e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:33.924707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe300b2-dff8-419e-99bd-24875906453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:33.925356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe300b2-dff8-419e-99bd-24875906453e None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:33.925641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe300b2-dff8-419e-99bd-24875906453e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:33.925918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe300b2-dff8-419e-99bd-24875906453e None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:33.926596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe300b2-dff8-419e-99bd-24875906453e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:33.927990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe300b2-dff8-419e-99bd-24875906453e None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:34.031317 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308149b3-95cc-445b-bedc-e2b38f537e9e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:34.068485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95c708-8de7-45c8-9521-7ddb0f3b32fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:34.069160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95c708-8de7-45c8-9521-7ddb0f3b32fa None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.069440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95c708-8de7-45c8-9521-7ddb0f3b32fa None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.069720 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95c708-8de7-45c8-9521-7ddb0f3b32fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:34.082154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95c708-8de7-45c8-9521-7ddb0f3b32fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:34.092463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95c708-8de7-45c8-9521-7ddb0f3b32fa None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-8f95c708-8de7-45c8-9521-7ddb0f3b32fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:56:34.100404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:34.101300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:56:34.102180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:56:34.102180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:56:34.102180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:56:34.102180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 17/33] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:56:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:34.102789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95c708-8de7-45c8-9521-7ddb0f3b32fa None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:34.155789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:34.219891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95c708-8de7-45c8-9521-7ddb0f3b32fa None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:34.220440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 17/34] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Aug 24 09:56:33 2022] POST /identity/v3/roles => generated 219 bytes in 545 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:34.221547 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe300b2-dff8-419e-99bd-24875906453e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:34.266703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:34.272919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3afacb-1189-4621-b297-d57f5cefae48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:34.278047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3afacb-1189-4621-b297-d57f5cefae48 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.278047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3afacb-1189-4621-b297-d57f5cefae48 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.278047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3afacb-1189-4621-b297-d57f5cefae48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:34.278047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3afacb-1189-4621-b297-d57f5cefae48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:34.278047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3afacb-1189-4621-b297-d57f5cefae48 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:34.280204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 18/35] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:56:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:34.281241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3afacb-1189-4621-b297-d57f5cefae48 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:34.350422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe300b2-dff8-419e-99bd-24875906453e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:34.403059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3afacb-1189-4621-b297-d57f5cefae48 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:34.403442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb26c5a-3972-442e-bf81-141bcb2e21b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:34.404220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26c5a-3972-442e-bf81-141bcb2e21b2 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.404632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26c5a-3972-442e-bf81-141bcb2e21b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.405041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26c5a-3972-442e-bf81-141bcb2e21b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:34.406020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb26c5a-3972-442e-bf81-141bcb2e21b2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:34.412218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb26c5a-3972-442e-bf81-141bcb2e21b2 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:34.423207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:34.446280 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb87e2-f958-4856-9d98-cb3c1623c383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:34.447706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb87e2-f958-4856-9d98-cb3c1623c383 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.448070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb87e2-f958-4856-9d98-cb3c1623c383 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.448455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb87e2-f958-4856-9d98-cb3c1623c383 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:34.449158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb87e2-f958-4856-9d98-cb3c1623c383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:34.449473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb87e2-f958-4856-9d98-cb3c1623c383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:34.451035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb87e2-f958-4856-9d98-cb3c1623c383 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:34.471294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 19/36] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:34.476261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb87e2-f958-4856-9d98-cb3c1623c383 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:34.566382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 18/37] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Aug 24 09:56:34 2022] POST /identity/v3/roles => generated 225 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:34.568166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fcd1f-535a-4136-b13c-b09c7735bd29 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.568509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fcd1f-535a-4136-b13c-b09c7735bd29 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.568938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fcd1f-535a-4136-b13c-b09c7735bd29 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:34.570825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 19/38] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:34.572205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fcd1f-535a-4136-b13c-b09c7735bd29 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:34.603642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb87e2-f958-4856-9d98-cb3c1623c383 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:34.654105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81c79c-6bb0-46cb-85c4-1c0fcfb01e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:34.654869 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81c79c-6bb0-46cb-85c4-1c0fcfb01e13 None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.655153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81c79c-6bb0-46cb-85c4-1c0fcfb01e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.655429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81c79c-6bb0-46cb-85c4-1c0fcfb01e13 None admin] PATH_INFO: `/v3/domains/default/users/deb4aa9488e54d0eafc940ddd57f345d/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:34.658459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81c79c-6bb0-46cb-85c4-1c0fcfb01e13 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=deb4aa9488e54d0eafc940ddd57f345d, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:34.660274 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81c79c-6bb0-46cb-85c4-1c0fcfb01e13 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:34.705674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:34.713690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fcd1f-535a-4136-b13c-b09c7735bd29 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:34.753584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed10ff-0d0c-4092-bdee-e2779269ffc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:34.754634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed10ff-0d0c-4092-bdee-e2779269ffc6 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.755126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed10ff-0d0c-4092-bdee-e2779269ffc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.755520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed10ff-0d0c-4092-bdee-e2779269ffc6 None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:34.756472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed10ff-0d0c-4092-bdee-e2779269ffc6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:34.757968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed10ff-0d0c-4092-bdee-e2779269ffc6 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:34.879674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 20/39] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:56:34 2022] POST /identity/v3/projects => generated 301 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:34.879674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d27478-84cd-4d0c-9392-6451f1134dda None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.879674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d27478-84cd-4d0c-9392-6451f1134dda None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.879674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d27478-84cd-4d0c-9392-6451f1134dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:34.903043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-83d27478-84cd-4d0c-9392-6451f1134dda None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:34.904909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-83d27478-84cd-4d0c-9392-6451f1134dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:34.983455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 20/40] 10.222.0.30 () {58 vars in 1361 bytes} [Wed Aug 24 09:56:34 2022] PUT /identity/v3/domains/default/users/deb4aa9488e54d0eafc940ddd57f345d/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:56:34.985044 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3673a24-eb5c-46bf-984c-5132e8d9ebb9 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.985348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3673a24-eb5c-46bf-984c-5132e8d9ebb9 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.985624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3673a24-eb5c-46bf-984c-5132e8d9ebb9 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:34.990846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 21/41] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:34.998436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de8730-7dcd-4814-9f61-8fe375c867cd None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.998799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de8730-7dcd-4814-9f61-8fe375c867cd None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.999006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de8730-7dcd-4814-9f61-8fe375c867cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:35.003250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 21/42] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:35.028788 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-95de8730-7dcd-4814-9f61-8fe375c867cd None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:35.031003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-95de8730-7dcd-4814-9f61-8fe375c867cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:35.071285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:35.119859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 22/43] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:56:35.285103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d27478-84cd-4d0c-9392-6451f1134dda None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:35.373624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:35.395857 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d27478-84cd-4d0c-9392-6451f1134dda None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:35.400742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de8730-7dcd-4814-9f61-8fe375c867cd None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:35.434101 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbcee7-08b3-488c-a387-6a27f1587738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:35.435133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbcee7-08b3-488c-a387-6a27f1587738 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:35.435420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbcee7-08b3-488c-a387-6a27f1587738 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:35.435699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbcee7-08b3-488c-a387-6a27f1587738 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:35.436401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbcee7-08b3-488c-a387-6a27f1587738 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:35.437851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbcee7-08b3-488c-a387-6a27f1587738 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:35.505935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de8730-7dcd-4814-9f61-8fe375c867cd None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:35.540730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd618e-2d85-4578-a7e7-f79e4bcf2572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:35.541832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd618e-2d85-4578-a7e7-f79e4bcf2572 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:35.542113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd618e-2d85-4578-a7e7-f79e4bcf2572 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:35.542447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd618e-2d85-4578-a7e7-f79e4bcf2572 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:35.543721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd618e-2d85-4578-a7e7-f79e4bcf2572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:35.545243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 22/44] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Aug 24 09:56:35 2022] POST /identity/v3/roles => generated 223 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:35.545470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd618e-2d85-4578-a7e7-f79e4bcf2572 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:35.547182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 23/45] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:56:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:35.553617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbcee7-08b3-488c-a387-6a27f1587738 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:35.673237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbcee7-08b3-488c-a387-6a27f1587738 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:35.713429 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281d565-a554-474d-99c9-554a4ea5181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:35.714075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281d565-a554-474d-99c9-554a4ea5181f None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:35.714364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281d565-a554-474d-99c9-554a4ea5181f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:35.714651 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281d565-a554-474d-99c9-554a4ea5181f None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:35.715324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281d565-a554-474d-99c9-554a4ea5181f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:35.716804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281d565-a554-474d-99c9-554a4ea5181f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:35.728598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:35.836515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 23/46] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:56:35 2022] POST /identity/v3/projects => generated 312 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:36.946122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cf8de-7ae4-4c7f-8b36-c6bd0c2c654f None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:36.946638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cf8de-7ae4-4c7f-8b36-c6bd0c2c654f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:36.946723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cf8de-7ae4-4c7f-8b36-c6bd0c2c654f None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:36.948087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 24/47] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:36.953583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b2380-e047-480b-8a4b-568be00295c4 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:36.954010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b2380-e047-480b-8a4b-568be00295c4 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:36.954131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b2380-e047-480b-8a4b-568be00295c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:36.972202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-021b2380-e047-480b-8a4b-568be00295c4 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:36.973419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-021b2380-e047-480b-8a4b-568be00295c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:37.044786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 24/48] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:37.315685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cf8de-7ae4-4c7f-8b36-c6bd0c2c654f None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:37.349968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:37.424475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cf8de-7ae4-4c7f-8b36-c6bd0c2c654f None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:37.459230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11940f05-be71-43d3-8fb2-0e6d393167b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:37.459869 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11940f05-be71-43d3-8fb2-0e6d393167b1 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:37.460154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11940f05-be71-43d3-8fb2-0e6d393167b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:37.460432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11940f05-be71-43d3-8fb2-0e6d393167b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:37.473460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11940f05-be71-43d3-8fb2-0e6d393167b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-11940f05-be71-43d3-8fb2-0e6d393167b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:37.478347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:37.479203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:56:37.480260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 25/49] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:56:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:37.485511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b2380-e047-480b-8a4b-568be00295c4 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:37.584576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b2380-e047-480b-8a4b-568be00295c4 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:37.605714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:37.619704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12c617-b306-434b-bcb6-4f58de071b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:37.620319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12c617-b306-434b-bcb6-4f58de071b52 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:37.620600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12c617-b306-434b-bcb6-4f58de071b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:37.620899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12c617-b306-434b-bcb6-4f58de071b52 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:37.621599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12c617-b306-434b-bcb6-4f58de071b52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:37.621868 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12c617-b306-434b-bcb6-4f58de071b52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:37.623242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12c617-b306-434b-bcb6-4f58de071b52 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:37.635895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 25/50] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:37.641545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11940f05-be71-43d3-8fb2-0e6d393167b1 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:37.734663 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:37.743141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11940f05-be71-43d3-8fb2-0e6d393167b1 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:37.760103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df85bc9-642e-4620-8a48-ba7de7ad9ed3 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:37.760493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df85bc9-642e-4620-8a48-ba7de7ad9ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:37.760706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df85bc9-642e-4620-8a48-ba7de7ad9ed3 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:37.762242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 26/51] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:56:37.767338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128923e-a721-4e81-904e-ff73ba5a5798 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:37.767698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128923e-a721-4e81-904e-ff73ba5a5798 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:37.767980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128923e-a721-4e81-904e-ff73ba5a5798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:37.782876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a756e5-42d5-43c6-a2b4-29abe296f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:37.783508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a756e5-42d5-43c6-a2b4-29abe296f6aa None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:37.783789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a756e5-42d5-43c6-a2b4-29abe296f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:37.784063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a756e5-42d5-43c6-a2b4-29abe296f6aa None admin] PATH_INFO: `/v3/users/deb4aa9488e54d0eafc940ddd57f345d` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:37.785203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a756e5-42d5-43c6-a2b4-29abe296f6aa None admin] RBAC: Authorizing `identity:get_user(user_id=deb4aa9488e54d0eafc940ddd57f345d)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:37.786290 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-b128923e-a721-4e81-904e-ff73ba5a5798 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:37.786648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a756e5-42d5-43c6-a2b4-29abe296f6aa None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:37.787491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b128923e-a721-4e81-904e-ff73ba5a5798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:37.788340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 26/52] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:37 2022] GET /identity/v3/users/deb4aa9488e54d0eafc940ddd57f345d => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:37.794878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a756e5-42d5-43c6-a2b4-29abe296f6aa None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:37.860219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 27/53] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:56:37.895742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a756e5-42d5-43c6-a2b4-29abe296f6aa None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:37.930855 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bc4e8-ec3c-427c-82ba-e0586e98f0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:37.931486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bc4e8-ec3c-427c-82ba-e0586e98f0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:37.931766 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bc4e8-ec3c-427c-82ba-e0586e98f0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:37.932039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bc4e8-ec3c-427c-82ba-e0586e98f0b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:37.933260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bc4e8-ec3c-427c-82ba-e0586e98f0b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:37.934669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bc4e8-ec3c-427c-82ba-e0586e98f0b3 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:37.936219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 27/54] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:56:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:37.942413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128923e-a721-4e81-904e-ff73ba5a5798 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:38.052243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128923e-a721-4e81-904e-ff73ba5a5798 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:38.065853 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:38.086988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5df7e-5177-4a43-bead-ae704ee28f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:38.088379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5df7e-5177-4a43-bead-ae704ee28f1b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:38.088910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5df7e-5177-4a43-bead-ae704ee28f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:38.089391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5df7e-5177-4a43-bead-ae704ee28f1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:38.090306 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5df7e-5177-4a43-bead-ae704ee28f1b None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.domain.id=default, user.id=deb4aa9488e54d0eafc940ddd57f345d) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:38.090623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5df7e-5177-4a43-bead-ae704ee28f1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:38.092137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5df7e-5177-4a43-bead-ae704ee28f1b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:38.101160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 28/55] 10.222.0.30 () {56 vars in 1322 bytes} [Wed Aug 24 09:56:37 2022] GET /identity/v3/role_assignments?user.id=deb4aa9488e54d0eafc940ddd57f345d&scope.domain.id=default&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:38.130232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bc4e8-ec3c-427c-82ba-e0586e98f0b3 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:38.162409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:38.240243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bc4e8-ec3c-427c-82ba-e0586e98f0b3 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:38.278123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e2ffb-12fd-486f-8531-90d39ade6d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:38.278763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e2ffb-12fd-486f-8531-90d39ade6d31 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:38.279042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e2ffb-12fd-486f-8531-90d39ade6d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:38.279314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e2ffb-12fd-486f-8531-90d39ade6d31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:38.280787 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e2ffb-12fd-486f-8531-90d39ade6d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:38.282268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e2ffb-12fd-486f-8531-90d39ade6d31 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:38.283829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 28/56] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:56:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:38.289622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5df7e-5177-4a43-bead-ae704ee28f1b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:38.394205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5df7e-5177-4a43-bead-ae704ee28f1b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:38.428676 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fc283-7833-452a-b404-8c9b72929ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:38.429313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fc283-7833-452a-b404-8c9b72929ac6 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:38.429598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fc283-7833-452a-b404-8c9b72929ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:38.429877 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fc283-7833-452a-b404-8c9b72929ac6 None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:38.430995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945fc283-7833-452a-b404-8c9b72929ac6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:38.432289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945fc283-7833-452a-b404-8c9b72929ac6 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:38.668577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 29/57] 10.222.0.30 () {60 vars in 1140 bytes} [Wed Aug 24 09:56:38 2022] POST /identity/v3/projects => generated 298 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:40.602012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531042d-0f14-47c0-8463-ab52703dba71 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:40.603052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531042d-0f14-47c0-8463-ab52703dba71 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:40.603052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531042d-0f14-47c0-8463-ab52703dba71 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:40.604501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 29/58] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:40.610153 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39f658-1b3d-4543-bc28-bdb48f4dc93d None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:40.610609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39f658-1b3d-4543-bc28-bdb48f4dc93d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:40.610707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39f658-1b3d-4543-bc28-bdb48f4dc93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:40.629057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-2e39f658-1b3d-4543-bc28-bdb48f4dc93d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:40.630309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2e39f658-1b3d-4543-bc28-bdb48f4dc93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:40.653872 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:40.699519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 30/59] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:40.977328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531042d-0f14-47c0-8463-ab52703dba71 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:41.074758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:41.081928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531042d-0f14-47c0-8463-ab52703dba71 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:41.117156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3424a7b-503c-4e49-b0de-ca9eba76bcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:41.118160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3424a7b-503c-4e49-b0de-ca9eba76bcfc None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:41.118436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3424a7b-503c-4e49-b0de-ca9eba76bcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:41.118716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3424a7b-503c-4e49-b0de-ca9eba76bcfc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:41.119867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3424a7b-503c-4e49-b0de-ca9eba76bcfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:41.121393 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3424a7b-503c-4e49-b0de-ca9eba76bcfc None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:41.123013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 30/60] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:56:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:41.128754 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39f658-1b3d-4543-bc28-bdb48f4dc93d None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:41.230022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39f658-1b3d-4543-bc28-bdb48f4dc93d None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:41.265783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4547b3-e471-4bd3-8d1a-00ebef1995de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:41.266622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4547b3-e471-4bd3-8d1a-00ebef1995de None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:41.266969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4547b3-e471-4bd3-8d1a-00ebef1995de None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:41.267342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4547b3-e471-4bd3-8d1a-00ebef1995de None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:41.268395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4547b3-e471-4bd3-8d1a-00ebef1995de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:41.270393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4547b3-e471-4bd3-8d1a-00ebef1995de None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:41.418609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 31/61] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:56:41 2022] POST /identity/v3/users => generated 275 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:44.417559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2a479-71fc-4a47-aeaf-17aad1a483db None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:44.418062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2a479-71fc-4a47-aeaf-17aad1a483db None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:44.418134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2a479-71fc-4a47-aeaf-17aad1a483db None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:44.419812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 31/62] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:44.426615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02bd61-6e60-40c9-8ca2-6b467bf1f1aa None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:44.427134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02bd61-6e60-40c9-8ca2-6b467bf1f1aa None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:44.427206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02bd61-6e60-40c9-8ca2-6b467bf1f1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:44.455141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1c02bd61-6e60-40c9-8ca2-6b467bf1f1aa None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:44.456367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1c02bd61-6e60-40c9-8ca2-6b467bf1f1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:44.469867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:44.556198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 32/63] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:44.952231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2a479-71fc-4a47-aeaf-17aad1a483db None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:44.964109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fe5ed-8568-48aa-b263-400d152fbdfe None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:44.964641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fe5ed-8568-48aa-b263-400d152fbdfe None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:44.965106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fe5ed-8568-48aa-b263-400d152fbdfe None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:44.966902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 33/64] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:44.974254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309cc7a-ffb9-481f-943a-32d3f08a2987 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:44.974730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309cc7a-ffb9-481f-943a-32d3f08a2987 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:44.975139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309cc7a-ffb9-481f-943a-32d3f08a2987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.022713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-3309cc7a-ffb9-481f-943a-32d3f08a2987 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:45.025201 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3309cc7a-ffb9-481f-943a-32d3f08a2987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:45.125217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 34/65] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:45.134442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2a479-71fc-4a47-aeaf-17aad1a483db None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:45.134856 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b443c-4623-4118-a0e2-ca41d2b9087b None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.135605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b443c-4623-4118-a0e2-ca41d2b9087b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.136069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b443c-4623-4118-a0e2-ca41d2b9087b None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.137724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 35/66] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:45 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:45.143763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d0c22-0764-42c0-bd1c-177278c085c7 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.144267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d0c22-0764-42c0-bd1c-177278c085c7 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.144700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d0c22-0764-42c0-bd1c-177278c085c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.181757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e551b-5eb1-4419-a096-88ec7f931191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:45.182765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e551b-5eb1-4419-a096-88ec7f931191 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.183261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e551b-5eb1-4419-a096-88ec7f931191 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.183730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e551b-5eb1-4419-a096-88ec7f931191 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.189993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-107d0c22-0764-42c0-bd1c-177278c085c7 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:45.191227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-107d0c22-0764-42c0-bd1c-177278c085c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:45.196078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e551b-5eb1-4419-a096-88ec7f931191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-019e551b-5eb1-4419-a096-88ec7f931191 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:45.204041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:45.204941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:56:45.205726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:56:45.208035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 32/67] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:56:45.218566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e551b-5eb1-4419-a096-88ec7f931191 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:45.247068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:45.347467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e551b-5eb1-4419-a096-88ec7f931191 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:45.390843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 36/68] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:45.392374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71275d08-ee70-42c7-ac95-98d6e0568715 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.392817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71275d08-ee70-42c7-ac95-98d6e0568715 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.393352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71275d08-ee70-42c7-ac95-98d6e0568715 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.394803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 37/69] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:45.404654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470f5d-d392-4f4b-bae1-adb2bc31e948 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.404654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470f5d-d392-4f4b-bae1-adb2bc31e948 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.404654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470f5d-d392-4f4b-bae1-adb2bc31e948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.412578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67e361-4335-4b1b-aa5c-8f08919093ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:45.413227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e361-4335-4b1b-aa5c-8f08919093ca None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.413507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e361-4335-4b1b-aa5c-8f08919093ca None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.413781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e361-4335-4b1b-aa5c-8f08919093ca None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.414460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67e361-4335-4b1b-aa5c-8f08919093ca None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:45.414769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67e361-4335-4b1b-aa5c-8f08919093ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:45.416688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67e361-4335-4b1b-aa5c-8f08919093ca None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:45.420698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:45.441249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-ef470f5d-d392-4f4b-bae1-adb2bc31e948 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:45.445933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 33/70] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:56:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:45.448878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ef470f5d-d392-4f4b-bae1-adb2bc31e948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:45.454810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67e361-4335-4b1b-aa5c-8f08919093ca None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:45.547311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 38/71] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:45.548950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d1954-dacf-492b-9993-f31936a0247b None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.549447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d1954-dacf-492b-9993-f31936a0247b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.549972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d1954-dacf-492b-9993-f31936a0247b None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.551485 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 39/72] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:45.552402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d1954-dacf-492b-9993-f31936a0247b None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:45.571701 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:45.590094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67e361-4335-4b1b-aa5c-8f08919093ca None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:45.636597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e7064-e034-4834-872f-a5e81d6fbc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:45.637674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e7064-e034-4834-872f-a5e81d6fbc0e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.637959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e7064-e034-4834-872f-a5e81d6fbc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.638246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e7064-e034-4834-872f-a5e81d6fbc0e None admin] PATH_INFO: `/v3/users/9d60ef5b46c84eaeaebaf462462afffb` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.658069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e7064-e034-4834-872f-a5e81d6fbc0e None admin] RBAC: Authorizing `identity:get_user(user_id=9d60ef5b46c84eaeaebaf462462afffb)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:45.660440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e7064-e034-4834-872f-a5e81d6fbc0e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:45.662471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 34/73] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:45 2022] GET /identity/v3/users/9d60ef5b46c84eaeaebaf462462afffb => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:45.664544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fc2f7-ff77-411e-a88a-4e3651fbc060 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.664944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fc2f7-ff77-411e-a88a-4e3651fbc060 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.665386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fc2f7-ff77-411e-a88a-4e3651fbc060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.686164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-fd7fc2f7-ff77-411e-a88a-4e3651fbc060 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:45.687738 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-fd7fc2f7-ff77-411e-a88a-4e3651fbc060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:45.691764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d1954-dacf-492b-9993-f31936a0247b None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:45.740740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242be2d-8ab2-4e70-af21-0b69465e6ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:45.741564 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242be2d-8ab2-4e70-af21-0b69465e6ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.741857 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242be2d-8ab2-4e70-af21-0b69465e6ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.742133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242be2d-8ab2-4e70-af21-0b69465e6ca6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.745066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:45.757301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242be2d-8ab2-4e70-af21-0b69465e6ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-a242be2d-8ab2-4e70-af21-0b69465e6ca6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:45.765170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:45.770630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 40/74] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:56:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:45.773783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242be2d-8ab2-4e70-af21-0b69465e6ca6 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:45.783139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 35/75] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:45.783814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fc2f7-ff77-411e-a88a-4e3651fbc060 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:45.866034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:45.900667 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242be2d-8ab2-4e70-af21-0b69465e6ca6 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:45.901034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fc2f7-ff77-411e-a88a-4e3651fbc060 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:45.938095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537a739-b579-4108-a904-1c964298556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:45.938731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537a739-b579-4108-a904-1c964298556c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.939017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537a739-b579-4108-a904-1c964298556c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.939296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537a739-b579-4108-a904-1c964298556c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.940502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da7bbc-c417-48f4-bb1d-af67129ec72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:45.941199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da7bbc-c417-48f4-bb1d-af67129ec72e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:45.941488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da7bbc-c417-48f4-bb1d-af67129ec72e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:45.941770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da7bbc-c417-48f4-bb1d-af67129ec72e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:45.953596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da7bbc-c417-48f4-bb1d-af67129ec72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-18da7bbc-c417-48f4-bb1d-af67129ec72e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:45.958323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:45.959152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 36/76] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:56:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:45.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da7bbc-c417-48f4-bb1d-af67129ec72e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:46.024666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:46.033336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537a739-b579-4108-a904-1c964298556c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:46.035074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537a739-b579-4108-a904-1c964298556c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:46.104373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da7bbc-c417-48f4-bb1d-af67129ec72e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:46.157479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3afb9-09c6-4aba-a98a-21764b2ef0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:46.158501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3afb9-09c6-4aba-a98a-21764b2ef0cc None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:46.158962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3afb9-09c6-4aba-a98a-21764b2ef0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:46.159458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3afb9-09c6-4aba-a98a-21764b2ef0cc None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:46.160729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3afb9-09c6-4aba-a98a-21764b2ef0cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:46.161291 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3afb9-09c6-4aba-a98a-21764b2ef0cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:46.163761 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3afb9-09c6-4aba-a98a-21764b2ef0cc None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:46.168295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 41/77] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:56:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:46.169203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537a739-b579-4108-a904-1c964298556c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:46.173401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:46.176745 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 37/78] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:56:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:46.177605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3afb9-09c6-4aba-a98a-21764b2ef0cc None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:46.305567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:46.312206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537a739-b579-4108-a904-1c964298556c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:46.326640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3afb9-09c6-4aba-a98a-21764b2ef0cc None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:46.347825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858a0f3-fd59-4951-8351-5a8ac091aca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:46.348636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858a0f3-fd59-4951-8351-5a8ac091aca9 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:46.349119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858a0f3-fd59-4951-8351-5a8ac091aca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:46.349770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858a0f3-fd59-4951-8351-5a8ac091aca9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:46.361514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858a0f3-fd59-4951-8351-5a8ac091aca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:46.364098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd1588-4046-4191-9383-110bc40e4e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:46.364704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd1588-4046-4191-9383-110bc40e4e95 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:46.365019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd1588-4046-4191-9383-110bc40e4e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:46.365304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd1588-4046-4191-9383-110bc40e4e95 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:46.365974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd1588-4046-4191-9383-110bc40e4e95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:46.366282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd1588-4046-4191-9383-110bc40e4e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-9858a0f3-fd59-4951-8351-5a8ac091aca9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:46.366711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:46.367526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:56:46.368314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:56:46.369537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd1588-4046-4191-9383-110bc40e4e95 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:46.369604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 42/79] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:46.369604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858a0f3-fd59-4951-8351-5a8ac091aca9 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:46.377082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 38/80] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:46.378017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd1588-4046-4191-9383-110bc40e4e95 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:46.437710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:46.480045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858a0f3-fd59-4951-8351-5a8ac091aca9 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:46.484486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd1588-4046-4191-9383-110bc40e4e95 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:46.516620 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9f93e-2d4d-4b9b-b8e1-d1bdb748c505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:46.517338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f93e-2d4d-4b9b-b8e1-d1bdb748c505 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:46.517626 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f93e-2d4d-4b9b-b8e1-d1bdb748c505 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:46.517906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f93e-2d4d-4b9b-b8e1-d1bdb748c505 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:46.519045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9f93e-2d4d-4b9b-b8e1-d1bdb748c505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:46.519801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd243-63d6-4ec2-a366-7d2ef6999235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:46.520441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd243-63d6-4ec2-a366-7d2ef6999235 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:46.520526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9f93e-2d4d-4b9b-b8e1-d1bdb748c505 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:46.520729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd243-63d6-4ec2-a366-7d2ef6999235 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:46.521049 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd243-63d6-4ec2-a366-7d2ef6999235 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:46.522157 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 43/81] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:56:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:46.522239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fd243-63d6-4ec2-a366-7d2ef6999235 None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:46.523637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fd243-63d6-4ec2-a366-7d2ef6999235 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:46.526018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 39/82] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:56:46 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:46.526880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd243-63d6-4ec2-a366-7d2ef6999235 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:46.528582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9f93e-2d4d-4b9b-b8e1-d1bdb748c505 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:46.554766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:46.641197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9f93e-2d4d-4b9b-b8e1-d1bdb748c505 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:46.651165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd243-63d6-4ec2-a366-7d2ef6999235 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:46.676774 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:46.684132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa49363-2603-4abf-b0e9-a9ab08014f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:46.684796 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa49363-2603-4abf-b0e9-a9ab08014f77 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:46.685132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa49363-2603-4abf-b0e9-a9ab08014f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:46.685414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa49363-2603-4abf-b0e9-a9ab08014f77 None admin] PATH_INFO: `/v3/users/9d60ef5b46c84eaeaebaf462462afffb` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:46.686625 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa82a0-215a-4ac7-9ee8-864e1124bd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:46.686955 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa49363-2603-4abf-b0e9-a9ab08014f77 None admin] RBAC: Authorizing `identity:get_user(user_id=9d60ef5b46c84eaeaebaf462462afffb)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:46.687228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa82a0-215a-4ac7-9ee8-864e1124bd84 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:46.687510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa82a0-215a-4ac7-9ee8-864e1124bd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:46.687785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa82a0-215a-4ac7-9ee8-864e1124bd84 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:46.688425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa49363-2603-4abf-b0e9-a9ab08014f77 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:46.688510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa82a0-215a-4ac7-9ee8-864e1124bd84 None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:46.688734 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa82a0-215a-4ac7-9ee8-864e1124bd84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:46.690093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 44/83] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:46 2022] GET /identity/v3/users/9d60ef5b46c84eaeaebaf462462afffb => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:46.694906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa82a0-215a-4ac7-9ee8-864e1124bd84 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:46.695019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa49363-2603-4abf-b0e9-a9ab08014f77 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:46.703399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 40/84] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:56:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:46.705344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa82a0-215a-4ac7-9ee8-864e1124bd84 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:46.812050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa82a0-215a-4ac7-9ee8-864e1124bd84 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:46.818431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:46.829497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa49363-2603-4abf-b0e9-a9ab08014f77 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:46.847884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2137f-f905-4f2f-9d27-538d5046a59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:46.848499 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2137f-f905-4f2f-9d27-538d5046a59b None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:46.848786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2137f-f905-4f2f-9d27-538d5046a59b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:46.849087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2137f-f905-4f2f-9d27-538d5046a59b None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:46.849734 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2137f-f905-4f2f-9d27-538d5046a59b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:46.851266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2137f-f905-4f2f-9d27-538d5046a59b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:46.863364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81761330-ed11-46ce-ac32-4864fe74ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:46.864410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81761330-ed11-46ce-ac32-4864fe74ea1b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:46.864948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81761330-ed11-46ce-ac32-4864fe74ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:46.866016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81761330-ed11-46ce-ac32-4864fe74ea1b None admin] PATH_INFO: `/v3/users/deb4aa9488e54d0eafc940ddd57f345d` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:46.869388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81761330-ed11-46ce-ac32-4864fe74ea1b None admin] RBAC: Authorizing `identity:get_user(user_id=deb4aa9488e54d0eafc940ddd57f345d)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:46.870875 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81761330-ed11-46ce-ac32-4864fe74ea1b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:46.872463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 45/85] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:46 2022] GET /identity/v3/users/deb4aa9488e54d0eafc940ddd57f345d => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:46.873258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81761330-ed11-46ce-ac32-4864fe74ea1b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:46.954624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 41/86] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:56:46 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:46.955617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2137f-f905-4f2f-9d27-538d5046a59b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:46.984532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81761330-ed11-46ce-ac32-4864fe74ea1b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:47.023344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:47.030635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e5339-eb00-4887-a823-40f136abf0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:47.031662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e5339-eb00-4887-a823-40f136abf0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:47.032194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e5339-eb00-4887-a823-40f136abf0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:47.032653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e5339-eb00-4887-a823-40f136abf0d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:47.033508 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e5339-eb00-4887-a823-40f136abf0d1 None admin] RBAC: Adding query filter params (role.id=13913230583d4ce384599ac5dace7ffa, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369, user.id=9d60ef5b46c84eaeaebaf462462afffb) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:47.033839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e5339-eb00-4887-a823-40f136abf0d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:47.035378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e5339-eb00-4887-a823-40f136abf0d1 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:47.050705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 46/87] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:56:46 2022] GET /identity/v3/role_assignments?user.id=9d60ef5b46c84eaeaebaf462462afffb&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=13913230583d4ce384599ac5dace7ffa => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:47.050705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e5339-eb00-4887-a823-40f136abf0d1 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:47.093499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:47.122305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2137f-f905-4f2f-9d27-538d5046a59b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:47.166142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062771c-acdd-4c5b-a72c-01c2622f79df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:47.166785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062771c-acdd-4c5b-a72c-01c2622f79df None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:47.167061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062771c-acdd-4c5b-a72c-01c2622f79df None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:47.167335 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062771c-acdd-4c5b-a72c-01c2622f79df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:47.200225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e5339-eb00-4887-a823-40f136abf0d1 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:47.253244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d2c36-8f4f-4abc-82b1-aad6ee7a0168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:47.254925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d2c36-8f4f-4abc-82b1-aad6ee7a0168 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:47.255387 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d2c36-8f4f-4abc-82b1-aad6ee7a0168 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:47.256131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d2c36-8f4f-4abc-82b1-aad6ee7a0168 None admin] PATH_INFO: `/v3/users/9d60ef5b46c84eaeaebaf462462afffb` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:47.258382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d2c36-8f4f-4abc-82b1-aad6ee7a0168 None admin] RBAC: Authorizing `identity:get_user(user_id=9d60ef5b46c84eaeaebaf462462afffb)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:47.260234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d2c36-8f4f-4abc-82b1-aad6ee7a0168 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:47.263382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 47/88] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:47 2022] GET /identity/v3/users/9d60ef5b46c84eaeaebaf462462afffb => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:47.264714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d2c36-8f4f-4abc-82b1-aad6ee7a0168 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:47.272432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062771c-acdd-4c5b-a72c-01c2622f79df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:47.274375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062771c-acdd-4c5b-a72c-01c2622f79df None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:47.371243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:47.398883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:47.418600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d2c36-8f4f-4abc-82b1-aad6ee7a0168 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:47.468588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 42/89] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:56:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:47.469541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062771c-acdd-4c5b-a72c-01c2622f79df None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:47.470777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfb6fd-de0b-40c0-a4b2-f467aa8dadc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:47.471426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb6fd-de0b-40c0-a4b2-f467aa8dadc8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:47.471715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb6fd-de0b-40c0-a4b2-f467aa8dadc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:47.471998 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb6fd-de0b-40c0-a4b2-f467aa8dadc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:47.562060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfb6fd-de0b-40c0-a4b2-f467aa8dadc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:47.563685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfb6fd-de0b-40c0-a4b2-f467aa8dadc8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:47.609316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062771c-acdd-4c5b-a72c-01c2622f79df None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:47.690435 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdf765-e0bb-4b26-a2c5-88523db935f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:47.691409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdf765-e0bb-4b26-a2c5-88523db935f4 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:47.691882 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdf765-e0bb-4b26-a2c5-88523db935f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:47.692347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdf765-e0bb-4b26-a2c5-88523db935f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:47.697200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:47.710126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 48/90] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:56:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:47.710126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfb6fd-de0b-40c0-a4b2-f467aa8dadc8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:47.719756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:47.819838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bdf765-e0bb-4b26-a2c5-88523db935f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:47.829629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bdf765-e0bb-4b26-a2c5-88523db935f4 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:47.834611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfb6fd-de0b-40c0-a4b2-f467aa8dadc8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:47.874851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff8924-cf22-467b-9e46-39a0ac5909da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:47.875648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff8924-cf22-467b-9e46-39a0ac5909da None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:47.875933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff8924-cf22-467b-9e46-39a0ac5909da None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:47.876213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff8924-cf22-467b-9e46-39a0ac5909da None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:47.877444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff8924-cf22-467b-9e46-39a0ac5909da None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:47.879042 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff8924-cf22-467b-9e46-39a0ac5909da None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:47.880925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 49/91] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:56:47 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:47.881697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff8924-cf22-467b-9e46-39a0ac5909da None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:47.953101 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:47.999729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff8924-cf22-467b-9e46-39a0ac5909da None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:48.037974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed74f9-877b-4d0f-b63c-a785b4a32b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:48.038618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed74f9-877b-4d0f-b63c-a785b4a32b3d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:48.038904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed74f9-877b-4d0f-b63c-a785b4a32b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:48.039181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed74f9-877b-4d0f-b63c-a785b4a32b3d None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:48.040363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed74f9-877b-4d0f-b63c-a785b4a32b3d None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:48.042210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed74f9-877b-4d0f-b63c-a785b4a32b3d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:48.043867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 50/92] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:56:47 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:48.044695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed74f9-877b-4d0f-b63c-a785b4a32b3d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:48.076163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 43/93] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:56:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:48.077151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdf765-e0bb-4b26-a2c5-88523db935f4 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:48.155159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:48.169442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed74f9-877b-4d0f-b63c-a785b4a32b3d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:48.196753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:48.206235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3ee33-1396-4642-98cd-58dc5d7dc77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:48.207067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3ee33-1396-4642-98cd-58dc5d7dc77c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:48.207530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3ee33-1396-4642-98cd-58dc5d7dc77c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:48.207970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3ee33-1396-4642-98cd-58dc5d7dc77c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:48.208874 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3ee33-1396-4642-98cd-58dc5d7dc77c None admin] RBAC: Adding query filter params (role.id=276ae07995d14b5d8ceb8e646392c2b1, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369, user.id=9d60ef5b46c84eaeaebaf462462afffb) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:48.209347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3ee33-1396-4642-98cd-58dc5d7dc77c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:48.210956 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3ee33-1396-4642-98cd-58dc5d7dc77c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:48.211061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdf765-e0bb-4b26-a2c5-88523db935f4 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:48.219852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 51/94] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:56:48 2022] GET /identity/v3/role_assignments?user.id=9d60ef5b46c84eaeaebaf462462afffb&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=276ae07995d14b5d8ceb8e646392c2b1 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:48.225182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3ee33-1396-4642-98cd-58dc5d7dc77c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:48.268974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31bcf2-68e6-442b-bb68-f1c9ccfcc579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:48.268974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31bcf2-68e6-442b-bb68-f1c9ccfcc579 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:48.268974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31bcf2-68e6-442b-bb68-f1c9ccfcc579 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:48.268974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31bcf2-68e6-442b-bb68-f1c9ccfcc579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:48.269381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31bcf2-68e6-442b-bb68-f1c9ccfcc579 None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369, user.id=deb4aa9488e54d0eafc940ddd57f345d) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:48.269381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31bcf2-68e6-442b-bb68-f1c9ccfcc579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:48.269381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31bcf2-68e6-442b-bb68-f1c9ccfcc579 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:48.274813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 44/95] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:56:48 2022] GET /identity/v3/role_assignments?user.id=deb4aa9488e54d0eafc940ddd57f345d&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:48.416326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3ee33-1396-4642-98cd-58dc5d7dc77c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:48.460784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8aa3b-01ee-4cfe-bb48-8aa30480d147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:48.461399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8aa3b-01ee-4cfe-bb48-8aa30480d147 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:48.462213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8aa3b-01ee-4cfe-bb48-8aa30480d147 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:48.462516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8aa3b-01ee-4cfe-bb48-8aa30480d147 None admin] PATH_INFO: `/v3/projects/e73480bacc274d19b1b2bb39dd93bc9b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:48.463751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8aa3b-01ee-4cfe-bb48-8aa30480d147 None admin] RBAC: Authorizing `identity:get_project(project_id=e73480bacc274d19b1b2bb39dd93bc9b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:48.465296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8aa3b-01ee-4cfe-bb48-8aa30480d147 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:48.467192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 52/96] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:56:48 2022] GET /identity/v3/projects/e73480bacc274d19b1b2bb39dd93bc9b => generated 312 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:48.473524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31bcf2-68e6-442b-bb68-f1c9ccfcc579 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:48.643483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31bcf2-68e6-442b-bb68-f1c9ccfcc579 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:48.655204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:48.693141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb7027-0ca8-4bde-bcdd-82559354aad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:48.694017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb7027-0ca8-4bde-bcdd-82559354aad1 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:48.694504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb7027-0ca8-4bde-bcdd-82559354aad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:48.695109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb7027-0ca8-4bde-bcdd-82559354aad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:48.695886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb7027-0ca8-4bde-bcdd-82559354aad1 None admin] RBAC: Adding query filter params (role.id=13913230583d4ce384599ac5dace7ffa, scope.project.id=e73480bacc274d19b1b2bb39dd93bc9b, user.id=9d60ef5b46c84eaeaebaf462462afffb) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:48.696944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb7027-0ca8-4bde-bcdd-82559354aad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:48.698495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb7027-0ca8-4bde-bcdd-82559354aad1 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:48.709323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 45/97] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:56:48 2022] GET /identity/v3/role_assignments?user.id=9d60ef5b46c84eaeaebaf462462afffb&scope.project.id=e73480bacc274d19b1b2bb39dd93bc9b&role.id=13913230583d4ce384599ac5dace7ffa => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:49.731245 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefb4ad-58a4-4c76-afde-2ed2776603df None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:49.732279 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefb4ad-58a4-4c76-afde-2ed2776603df None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:49.732782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefb4ad-58a4-4c76-afde-2ed2776603df None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:49.734394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 53/98] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:49.740100 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ec2f5-388e-4ef1-8b2e-fa58dd1f4b10 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:49.740991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ec2f5-388e-4ef1-8b2e-fa58dd1f4b10 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:49.741469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ec2f5-388e-4ef1-8b2e-fa58dd1f4b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:49.761156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-379ec2f5-388e-4ef1-8b2e-fa58dd1f4b10 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:49.762713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-379ec2f5-388e-4ef1-8b2e-fa58dd1f4b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:49.852251 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 46/99] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:56:49.923078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4b255-2526-411a-8775-47e999e25300 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:49.923789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4b255-2526-411a-8775-47e999e25300 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:49.924279 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4b255-2526-411a-8775-47e999e25300 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:49.925879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 54/100] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:49.931912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e6c96-029a-4473-82d8-f071af50bea3 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:49.932208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e6c96-029a-4473-82d8-f071af50bea3 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:49.932495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e6c96-029a-4473-82d8-f071af50bea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:49.951543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-005e6c96-029a-4473-82d8-f071af50bea3 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:49.952771 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-005e6c96-029a-4473-82d8-f071af50bea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:50.042480 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 47/101] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:50.278567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dae9ee-6e9c-4738-8d43-b5e48ff96eb0 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:50.278947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dae9ee-6e9c-4738-8d43-b5e48ff96eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:50.279361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dae9ee-6e9c-4738-8d43-b5e48ff96eb0 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:50.281065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 55/102] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:50.295433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e20de8-1567-4e9e-9e47-176da7534c85 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:50.295433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e20de8-1567-4e9e-9e47-176da7534c85 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:50.296171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e20de8-1567-4e9e-9e47-176da7534c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:50.322373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-07e20de8-1567-4e9e-9e47-176da7534c85 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:50.323570 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-07e20de8-1567-4e9e-9e47-176da7534c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:50.360879 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:50.409794 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dae9ee-6e9c-4738-8d43-b5e48ff96eb0 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:50.442217 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 48/103] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:50.488587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e20de8-1567-4e9e-9e47-176da7534c85 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:50.512890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:50.581198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dae9ee-6e9c-4738-8d43-b5e48ff96eb0 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:50.657583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b85836-5bd4-48dd-9d5a-64c66b5e44cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:50.658700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b85836-5bd4-48dd-9d5a-64c66b5e44cd None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:50.659210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b85836-5bd4-48dd-9d5a-64c66b5e44cd None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:50.659707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b85836-5bd4-48dd-9d5a-64c66b5e44cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:50.689748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b85836-5bd4-48dd-9d5a-64c66b5e44cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-00b85836-5bd4-48dd-9d5a-64c66b5e44cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:50.700843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:50.701940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:56:50.702729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:56:50.703340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 56/104] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:56:50.717513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b85836-5bd4-48dd-9d5a-64c66b5e44cd None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:50.723035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e20de8-1567-4e9e-9e47-176da7534c85 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:50.792582 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae58286-12f8-4201-b0a1-2c709e2b05c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:50.793659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae58286-12f8-4201-b0a1-2c709e2b05c0 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:50.794139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae58286-12f8-4201-b0a1-2c709e2b05c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:50.794608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae58286-12f8-4201-b0a1-2c709e2b05c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:50.796458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae58286-12f8-4201-b0a1-2c709e2b05c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:50.799617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae58286-12f8-4201-b0a1-2c709e2b05c0 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:50.802314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 49/105] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:56:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:50.810356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae58286-12f8-4201-b0a1-2c709e2b05c0 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:50.868236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:50.971353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae58286-12f8-4201-b0a1-2c709e2b05c0 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:51.017097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab031e7-7247-46a5-bcca-9d69d427a71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:51.017931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab031e7-7247-46a5-bcca-9d69d427a71c None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:51.018390 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab031e7-7247-46a5-bcca-9d69d427a71c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:51.018761 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab031e7-7247-46a5-bcca-9d69d427a71c None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:51.019567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab031e7-7247-46a5-bcca-9d69d427a71c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:51.021596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab031e7-7247-46a5-bcca-9d69d427a71c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:51.029330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b85836-5bd4-48dd-9d5a-64c66b5e44cd None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:51.063134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:51.081121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dffbf-200b-43c4-a521-f6e722dc3561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:51.081788 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dffbf-200b-43c4-a521-f6e722dc3561 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:51.082082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dffbf-200b-43c4-a521-f6e722dc3561 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:51.082364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dffbf-200b-43c4-a521-f6e722dc3561 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:51.083038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7dffbf-200b-43c4-a521-f6e722dc3561 None admin] RBAC: Adding query filter params (name=member) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:51.083349 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7dffbf-200b-43c4-a521-f6e722dc3561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:51.084830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7dffbf-200b-43c4-a521-f6e722dc3561 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:51.095110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 57/106] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:56:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:51.095984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dffbf-200b-43c4-a521-f6e722dc3561 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:51.135784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 50/107] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:56:50 2022] POST /identity/v3/projects => generated 302 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:51.136404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab031e7-7247-46a5-bcca-9d69d427a71c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:51.333458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dffbf-200b-43c4-a521-f6e722dc3561 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:51.337193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:51.377920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab031e7-7247-46a5-bcca-9d69d427a71c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:51.386037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007863a8-4ebe-4216-9d86-106fe00a03c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:51.386656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007863a8-4ebe-4216-9d86-106fe00a03c7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:51.386946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007863a8-4ebe-4216-9d86-106fe00a03c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:51.387254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007863a8-4ebe-4216-9d86-106fe00a03c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:51.412949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007863a8-4ebe-4216-9d86-106fe00a03c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-007863a8-4ebe-4216-9d86-106fe00a03c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:51.420094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:51.421654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:56:51.423013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 58/108] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:51.432669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007863a8-4ebe-4216-9d86-106fe00a03c7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:51.460012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37621e-2c56-4aa4-ae3f-77ed4885d354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:51.460254 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37621e-2c56-4aa4-ae3f-77ed4885d354 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:51.460488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37621e-2c56-4aa4-ae3f-77ed4885d354 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:51.460797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37621e-2c56-4aa4-ae3f-77ed4885d354 None admin] PATH_INFO: `/v3/users/9d60ef5b46c84eaeaebaf462462afffb` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:51.462350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37621e-2c56-4aa4-ae3f-77ed4885d354 None admin] RBAC: Authorizing `identity:get_user(user_id=9d60ef5b46c84eaeaebaf462462afffb)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:51.464043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37621e-2c56-4aa4-ae3f-77ed4885d354 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:51.466040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 51/109] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:51 2022] GET /identity/v3/users/9d60ef5b46c84eaeaebaf462462afffb => generated 275 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:51.472787 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37621e-2c56-4aa4-ae3f-77ed4885d354 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:51.528427 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:51.613852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007863a8-4ebe-4216-9d86-106fe00a03c7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:51.630116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37621e-2c56-4aa4-ae3f-77ed4885d354 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:51.658849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6425f53-85bc-4f2f-a58c-675e68ed1738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:51.659397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6425f53-85bc-4f2f-a58c-675e68ed1738 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:51.659681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6425f53-85bc-4f2f-a58c-675e68ed1738 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:51.659952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6425f53-85bc-4f2f-a58c-675e68ed1738 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:51.661040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6425f53-85bc-4f2f-a58c-675e68ed1738 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:51.661354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6425f53-85bc-4f2f-a58c-675e68ed1738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:51.663050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6425f53-85bc-4f2f-a58c-675e68ed1738 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:51.672794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0ba15-2aa9-4a4a-a0a7-eb773a081256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:51.673798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0ba15-2aa9-4a4a-a0a7-eb773a081256 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:51.674340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0ba15-2aa9-4a4a-a0a7-eb773a081256 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:51.674495 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 59/110] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:56:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:51.675514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0ba15-2aa9-4a4a-a0a7-eb773a081256 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:51.676950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4e4d4-8f04-4296-849d-9bf156a960c2 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:51.676950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4e4d4-8f04-4296-849d-9bf156a960c2 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:51.679025 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0ba15-2aa9-4a4a-a0a7-eb773a081256 None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:51.679234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4e4d4-8f04-4296-849d-9bf156a960c2 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:51.681348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 60/111] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:51.682030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0ba15-2aa9-4a4a-a0a7-eb773a081256 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:51.682361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d9f7f-7a6a-4a3b-8301-46ef30fa48a4 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:51.682745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d9f7f-7a6a-4a3b-8301-46ef30fa48a4 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:51.683030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d9f7f-7a6a-4a3b-8301-46ef30fa48a4 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:51.684556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 61/112] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:51.685015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 52/113] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:56:51 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:51.686693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2c74b-44d1-4330-b5e3-2e2f0349e391 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:51.687188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2c74b-44d1-4330-b5e3-2e2f0349e391 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:51.687655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2c74b-44d1-4330-b5e3-2e2f0349e391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:51.692054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d9f7f-7a6a-4a3b-8301-46ef30fa48a4 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:51.714153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-fbf2c74b-44d1-4330-b5e3-2e2f0349e391 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:51.715693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-fbf2c74b-44d1-4330-b5e3-2e2f0349e391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:51.731842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:51.803019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 53/114] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 09:56:51.805867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbab48-f512-461c-889f-6814d61efd20 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:51.806619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbab48-f512-461c-889f-6814d61efd20 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:51.807230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbab48-f512-461c-889f-6814d61efd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:51.818547 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d9f7f-7a6a-4a3b-8301-46ef30fa48a4 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:51.822109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:51.844568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-08fbab48-f512-461c-889f-6814d61efd20 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:51.846093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-08fbab48-f512-461c-889f-6814d61efd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:51.868998 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8aea2-8095-490b-9a28-c3ee4544557f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:51.869448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8aea2-8095-490b-9a28-c3ee4544557f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:51.869730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8aea2-8095-490b-9a28-c3ee4544557f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:51.870074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8aea2-8095-490b-9a28-c3ee4544557f None admin] PATH_INFO: `/v3/users/02a015006c5346118e582f84a4753f25` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:51.891500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8aea2-8095-490b-9a28-c3ee4544557f None admin] RBAC: Authorizing `identity:get_user(user_id=02a015006c5346118e582f84a4753f25)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:51.893498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8aea2-8095-490b-9a28-c3ee4544557f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:51.895446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 62/115] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:51 2022] GET /identity/v3/users/02a015006c5346118e582f84a4753f25 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:51.897221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8aea2-8095-490b-9a28-c3ee4544557f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:51.945665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 54/116] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:51.948306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbab48-f512-461c-889f-6814d61efd20 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:52.056642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:52.095599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbab48-f512-461c-889f-6814d61efd20 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:52.101921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8aea2-8095-490b-9a28-c3ee4544557f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:52.179465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b8bc4-84a3-437c-aa49-ab4db08716eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:52.180141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b8bc4-84a3-437c-aa49-ab4db08716eb None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:52.180620 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b8bc4-84a3-437c-aa49-ab4db08716eb None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:52.180977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b8bc4-84a3-437c-aa49-ab4db08716eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:52.189581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8cffb-7d18-431e-aed8-39a716758158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:52.190713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8cffb-7d18-431e-aed8-39a716758158 None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:52.191223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8cffb-7d18-431e-aed8-39a716758158 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:52.191702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8cffb-7d18-431e-aed8-39a716758158 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/users/9d60ef5b46c84eaeaebaf462462afffb/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:52.205493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8cffb-7d18-431e-aed8-39a716758158 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4bdbfd07cd94eeea5087a9d39f83369, user_id=9d60ef5b46c84eaeaebaf462462afffb, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:52.207582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8cffb-7d18-431e-aed8-39a716758158 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:52.293631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 63/117] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:56:51 2022] PUT /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/users/9d60ef5b46c84eaeaebaf462462afffb/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:56:52.296329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf742-3c8a-46d3-acde-6533f7fe086e None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:52.297614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf742-3c8a-46d3-acde-6533f7fe086e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:52.298136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf742-3c8a-46d3-acde-6533f7fe086e None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:52.300010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 64/118] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:52.301138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cf742-3c8a-46d3-acde-6533f7fe086e None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:52.322962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b8bc4-84a3-437c-aa49-ab4db08716eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:52.324596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b8bc4-84a3-437c-aa49-ab4db08716eb None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:52.333484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:52.481637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:52.513896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 55/119] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:56:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:52.515131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa81f1-56b6-42d9-8b21-d9137fa4be09 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:52.515432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa81f1-56b6-42d9-8b21-d9137fa4be09 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:52.515719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa81f1-56b6-42d9-8b21-d9137fa4be09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:52.551500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e9aa81f1-56b6-42d9-8b21-d9137fa4be09 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:52.553065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e9aa81f1-56b6-42d9-8b21-d9137fa4be09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:52.584918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cf742-3c8a-46d3-acde-6533f7fe086e None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:52.640340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a81e2-93ff-4edf-9357-73a11ab10861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:52.642951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a81e2-93ff-4edf-9357-73a11ab10861 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:52.644413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a81e2-93ff-4edf-9357-73a11ab10861 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:52.645514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a81e2-93ff-4edf-9357-73a11ab10861 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:52.663152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 56/120] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:52.667238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a81e2-93ff-4edf-9357-73a11ab10861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:52.670127 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa81f1-56b6-42d9-8b21-d9137fa4be09 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-d27a81e2-93ff-4edf-9357-73a11ab10861 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:52.675197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:52.676238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:52.677076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:56:52.681095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 65/121] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:56:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:52.682677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a81e2-93ff-4edf-9357-73a11ab10861 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:52.813599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:52.843120 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a81e2-93ff-4edf-9357-73a11ab10861 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:52.898091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41739d1e-567b-4466-8e9e-d2cc9e5aa85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:52.900173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41739d1e-567b-4466-8e9e-d2cc9e5aa85f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:52.900825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41739d1e-567b-4466-8e9e-d2cc9e5aa85f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:52.901361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41739d1e-567b-4466-8e9e-d2cc9e5aa85f None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:52.904247 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41739d1e-567b-4466-8e9e-d2cc9e5aa85f None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:52.906229 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41739d1e-567b-4466-8e9e-d2cc9e5aa85f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:52.909963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 66/122] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:56:52 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:52.911790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41739d1e-567b-4466-8e9e-d2cc9e5aa85f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:52.969727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa81f1-56b6-42d9-8b21-d9137fa4be09 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:52.991709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:53.061952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41739d1e-567b-4466-8e9e-d2cc9e5aa85f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:53.077717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde765a-f50d-407c-9e45-3c70a1861865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:53.078282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde765a-f50d-407c-9e45-3c70a1861865 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.078589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde765a-f50d-407c-9e45-3c70a1861865 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.079229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde765a-f50d-407c-9e45-3c70a1861865 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.095988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde765a-f50d-407c-9e45-3c70a1861865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-dfde765a-f50d-407c-9e45-3c70a1861865 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:53.100981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:53.101913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:56:53.102668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 57/123] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:56:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:53.104272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde765a-f50d-407c-9e45-3c70a1861865 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:53.125431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bc8a2-30d2-4061-9cdc-cbad41264d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:53.127537 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bc8a2-30d2-4061-9cdc-cbad41264d8e None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.128096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bc8a2-30d2-4061-9cdc-cbad41264d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.129027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bc8a2-30d2-4061-9cdc-cbad41264d8e None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.130446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bc8a2-30d2-4061-9cdc-cbad41264d8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:53.131241 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bc8a2-30d2-4061-9cdc-cbad41264d8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:53.135233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bc8a2-30d2-4061-9cdc-cbad41264d8e None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:53.149472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 67/124] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:53.149472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bc8a2-30d2-4061-9cdc-cbad41264d8e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:53.176971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:53.239226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde765a-f50d-407c-9e45-3c70a1861865 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:53.283083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63282f9a-499f-4e3e-80ea-ac9682974d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:53.284166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282f9a-499f-4e3e-80ea-ac9682974d94 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.284676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282f9a-499f-4e3e-80ea-ac9682974d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.285187 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282f9a-499f-4e3e-80ea-ac9682974d94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.286333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63282f9a-499f-4e3e-80ea-ac9682974d94 None admin] RBAC: Adding query filter params (role.id=b9b25895899d46eea0c912f6277165cb, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369, user.id=02a015006c5346118e582f84a4753f25) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:53.286758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63282f9a-499f-4e3e-80ea-ac9682974d94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:53.289295 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63282f9a-499f-4e3e-80ea-ac9682974d94 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:53.309381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 58/125] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:56:53 2022] GET /identity/v3/role_assignments?user.id=02a015006c5346118e582f84a4753f25&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=b9b25895899d46eea0c912f6277165cb => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:53.310412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63282f9a-499f-4e3e-80ea-ac9682974d94 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:53.322136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bc8a2-30d2-4061-9cdc-cbad41264d8e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:53.376458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:53.381203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d94967-8aeb-4fdb-ab79-8fc76cd803e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:53.382083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d94967-8aeb-4fdb-ab79-8fc76cd803e0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.382452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d94967-8aeb-4fdb-ab79-8fc76cd803e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.382895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d94967-8aeb-4fdb-ab79-8fc76cd803e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.404490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d94967-8aeb-4fdb-ab79-8fc76cd803e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-60d94967-8aeb-4fdb-ab79-8fc76cd803e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:53.417712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:53.418729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:56:53.419519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 68/126] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:53.420169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d94967-8aeb-4fdb-ab79-8fc76cd803e0 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:53.484885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63282f9a-499f-4e3e-80ea-ac9682974d94 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:53.524502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68a431-15c3-46f1-a518-f178b4b514c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:53.525887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68a431-15c3-46f1-a518-f178b4b514c4 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.526400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68a431-15c3-46f1-a518-f178b4b514c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.526876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68a431-15c3-46f1-a518-f178b4b514c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.527808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68a431-15c3-46f1-a518-f178b4b514c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:53.528361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68a431-15c3-46f1-a518-f178b4b514c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:53.529986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68a431-15c3-46f1-a518-f178b4b514c4 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:53.541478 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 59/127] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:56:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:53.542722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68a431-15c3-46f1-a518-f178b4b514c4 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:53.588725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:53.645621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d94967-8aeb-4fdb-ab79-8fc76cd803e0 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:53.705430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec19d42-0fe5-49f8-be1d-a294c38d4950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:53.706134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec19d42-0fe5-49f8-be1d-a294c38d4950 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.706457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec19d42-0fe5-49f8-be1d-a294c38d4950 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.706772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec19d42-0fe5-49f8-be1d-a294c38d4950 None admin] PATH_INFO: `/v3/users/deb4aa9488e54d0eafc940ddd57f345d` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.708167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec19d42-0fe5-49f8-be1d-a294c38d4950 None admin] RBAC: Authorizing `identity:get_user(user_id=deb4aa9488e54d0eafc940ddd57f345d)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:53.710184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec19d42-0fe5-49f8-be1d-a294c38d4950 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:53.711930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:53.714471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 69/128] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:53 2022] GET /identity/v3/users/deb4aa9488e54d0eafc940ddd57f345d => generated 247 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:53.715365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec19d42-0fe5-49f8-be1d-a294c38d4950 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:53.720972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68a431-15c3-46f1-a518-f178b4b514c4 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:53.772729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc635242-f7ab-43d8-a55e-186e8cd951c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:53.773641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc635242-f7ab-43d8-a55e-186e8cd951c0 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.773969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc635242-f7ab-43d8-a55e-186e8cd951c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.774962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc635242-f7ab-43d8-a55e-186e8cd951c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.775667 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc635242-f7ab-43d8-a55e-186e8cd951c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:53.775976 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc635242-f7ab-43d8-a55e-186e8cd951c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:53.777579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc635242-f7ab-43d8-a55e-186e8cd951c0 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:53.794420 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 60/129] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:56:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:53.795717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fa3bd-1704-497e-a03a-76bae9d6fe1d None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.796010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fa3bd-1704-497e-a03a-76bae9d6fe1d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.800904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fa3bd-1704-497e-a03a-76bae9d6fe1d None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.802596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 61/130] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:53.808302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fa3bd-1704-497e-a03a-76bae9d6fe1d None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:53.885894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec19d42-0fe5-49f8-be1d-a294c38d4950 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:53.955490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7eabcd-277e-47d6-9eac-b0aae0d432db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:53.957239 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7eabcd-277e-47d6-9eac-b0aae0d432db None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.958126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7eabcd-277e-47d6-9eac-b0aae0d432db None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.958815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7eabcd-277e-47d6-9eac-b0aae0d432db None admin] PATH_INFO: `/v3/users/9d60ef5b46c84eaeaebaf462462afffb` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.960906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7eabcd-277e-47d6-9eac-b0aae0d432db None admin] RBAC: Authorizing `identity:get_user(user_id=9d60ef5b46c84eaeaebaf462462afffb)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:53.961485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fa3bd-1704-497e-a03a-76bae9d6fe1d None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:53.963897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7eabcd-277e-47d6-9eac-b0aae0d432db None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:53.968313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 70/131] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:53 2022] GET /identity/v3/users/9d60ef5b46c84eaeaebaf462462afffb => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:53.970176 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7eabcd-277e-47d6-9eac-b0aae0d432db None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:53.990137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:54.046480 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a60e2-8242-4613-b65c-43d47e5b73b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:54.048300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a60e2-8242-4613-b65c-43d47e5b73b1 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:54.049394 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a60e2-8242-4613-b65c-43d47e5b73b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:54.050136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a60e2-8242-4613-b65c-43d47e5b73b1 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:54.053264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a60e2-8242-4613-b65c-43d47e5b73b1 None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:54.057058 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a60e2-8242-4613-b65c-43d47e5b73b1 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:54.060379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 62/132] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:56:53 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:54.063581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff70ac-0d6f-4935-93a9-9f9de41f93b8 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:54.065108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff70ac-0d6f-4935-93a9-9f9de41f93b8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:54.065866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff70ac-0d6f-4935-93a9-9f9de41f93b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:54.106869 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-d4ff70ac-0d6f-4935-93a9-9f9de41f93b8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:54.110142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d4ff70ac-0d6f-4935-93a9-9f9de41f93b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:54.153045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:54.155345 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7eabcd-277e-47d6-9eac-b0aae0d432db None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:54.210174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa072f8-3a89-48de-9e9a-07db2bed5f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:54.211414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa072f8-3a89-48de-9e9a-07db2bed5f36 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:54.212246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa072f8-3a89-48de-9e9a-07db2bed5f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:54.212526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa072f8-3a89-48de-9e9a-07db2bed5f36 None admin] PATH_INFO: `/v3/users/9d60ef5b46c84eaeaebaf462462afffb` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:54.214984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa072f8-3a89-48de-9e9a-07db2bed5f36 None admin] RBAC: Authorizing `identity:get_user(user_id=9d60ef5b46c84eaeaebaf462462afffb)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:54.217999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa072f8-3a89-48de-9e9a-07db2bed5f36 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:54.220445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 63/133] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:54.221593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff70ac-0d6f-4935-93a9-9f9de41f93b8 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:54.226304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 71/134] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:53 2022] GET /identity/v3/users/9d60ef5b46c84eaeaebaf462462afffb => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:54.227791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa072f8-3a89-48de-9e9a-07db2bed5f36 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:54.355432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:54.382664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa072f8-3a89-48de-9e9a-07db2bed5f36 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:54.419183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0ee33-a248-4b7d-978b-fb1cfed99daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:54.419863 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ee33-a248-4b7d-978b-fb1cfed99daa None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:54.420204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ee33-a248-4b7d-978b-fb1cfed99daa None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:54.420497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ee33-a248-4b7d-978b-fb1cfed99daa None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/users/deb4aa9488e54d0eafc940ddd57f345d/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:54.422535 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0ee33-a248-4b7d-978b-fb1cfed99daa None admin] RBAC: Authorizing `identity:create_grant(project_id=c4bdbfd07cd94eeea5087a9d39f83369, user_id=deb4aa9488e54d0eafc940ddd57f345d, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:54.423998 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0ee33-a248-4b7d-978b-fb1cfed99daa None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:54.447864 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff70ac-0d6f-4935-93a9-9f9de41f93b8 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:54.501843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9bbf1-2df7-4844-84bf-2b7f3eaeb746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:54.502781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9bbf1-2df7-4844-84bf-2b7f3eaeb746 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:54.503246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9bbf1-2df7-4844-84bf-2b7f3eaeb746 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:54.503787 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9bbf1-2df7-4844-84bf-2b7f3eaeb746 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:54.505691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a9bbf1-2df7-4844-84bf-2b7f3eaeb746 None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:54.518125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a9bbf1-2df7-4844-84bf-2b7f3eaeb746 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:54.518125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 64/135] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:56:54 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:54.518125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9bbf1-2df7-4844-84bf-2b7f3eaeb746 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:54.552945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:54.556041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 72/136] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:56:54 2022] PUT /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/users/deb4aa9488e54d0eafc940ddd57f345d/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:56:54.556976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0ee33-a248-4b7d-978b-fb1cfed99daa None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:54.695465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9bbf1-2df7-4844-84bf-2b7f3eaeb746 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:54.738801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2ac88-b547-4b2e-af83-04939a5af9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:54.739762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2ac88-b547-4b2e-af83-04939a5af9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:54.740223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2ac88-b547-4b2e-af83-04939a5af9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:54.740704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2ac88-b547-4b2e-af83-04939a5af9e5 None admin] PATH_INFO: `/v3/projects/e73480bacc274d19b1b2bb39dd93bc9b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:54.743694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2ac88-b547-4b2e-af83-04939a5af9e5 None admin] RBAC: Authorizing `identity:get_project(project_id=e73480bacc274d19b1b2bb39dd93bc9b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:54.745527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2ac88-b547-4b2e-af83-04939a5af9e5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:54.747512 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 65/137] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:56:54 2022] GET /identity/v3/projects/e73480bacc274d19b1b2bb39dd93bc9b => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:54.748568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2ac88-b547-4b2e-af83-04939a5af9e5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:54.844555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0ee33-a248-4b7d-978b-fb1cfed99daa None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:54.888071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:54.897256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2ac88-b547-4b2e-af83-04939a5af9e5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:54.905636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fe907-f15b-428e-aca9-bb531f0d6190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:54.907090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fe907-f15b-428e-aca9-bb531f0d6190 None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:54.907819 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fe907-f15b-428e-aca9-bb531f0d6190 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:54.908502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fe907-f15b-428e-aca9-bb531f0d6190 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/users/9d60ef5b46c84eaeaebaf462462afffb/roles/276ae07995d14b5d8ceb8e646392c2b1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:54.913102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fe907-f15b-428e-aca9-bb531f0d6190 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4bdbfd07cd94eeea5087a9d39f83369, user_id=9d60ef5b46c84eaeaebaf462462afffb, role_id=276ae07995d14b5d8ceb8e646392c2b1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:54.916232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fe907-f15b-428e-aca9-bb531f0d6190 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:54.919011 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:54.951390 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b36622-83b3-4690-9ce1-b6eb9736cc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:54.952113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b36622-83b3-4690-9ce1-b6eb9736cc92 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:54.952587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b36622-83b3-4690-9ce1-b6eb9736cc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:54.952908 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b36622-83b3-4690-9ce1-b6eb9736cc92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:54.954316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b36622-83b3-4690-9ce1-b6eb9736cc92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:54.957310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b36622-83b3-4690-9ce1-b6eb9736cc92 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:54.960145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 66/138] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:56:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:54.960628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b36622-83b3-4690-9ce1-b6eb9736cc92 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:55.022089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 73/139] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:56:54 2022] PUT /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/users/9d60ef5b46c84eaeaebaf462462afffb/roles/276ae07995d14b5d8ceb8e646392c2b1 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:56:55.024684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2d873-28ab-446b-9043-bd0dde0925e3 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:55.026431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2d873-28ab-446b-9043-bd0dde0925e3 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:55.027071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2d873-28ab-446b-9043-bd0dde0925e3 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:55.029574 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 74/140] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:55.031139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2d873-28ab-446b-9043-bd0dde0925e3 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:55.139809 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b36622-83b3-4690-9ce1-b6eb9736cc92 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:55.208486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb3d06-9abd-4113-a93b-70d86de15ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:55.209751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb3d06-9abd-4113-a93b-70d86de15ab6 None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:55.210584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb3d06-9abd-4113-a93b-70d86de15ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:55.211102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb3d06-9abd-4113-a93b-70d86de15ab6 None admin] PATH_INFO: `/v3/projects/e73480bacc274d19b1b2bb39dd93bc9b/users/9d60ef5b46c84eaeaebaf462462afffb/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:55.214540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb3d06-9abd-4113-a93b-70d86de15ab6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e73480bacc274d19b1b2bb39dd93bc9b, user_id=9d60ef5b46c84eaeaebaf462462afffb, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:55.216423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb3d06-9abd-4113-a93b-70d86de15ab6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:55.240396 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2d873-28ab-446b-9043-bd0dde0925e3 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:55.302998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 67/141] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:56:54 2022] PUT /identity/v3/projects/e73480bacc274d19b1b2bb39dd93bc9b/users/9d60ef5b46c84eaeaebaf462462afffb/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:56:55.306695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e815ae1-a24e-47cf-9c6b-147705baa36b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:55.307637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab32cc-54df-45c7-aae8-3b1be1ef7075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:55.308144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e815ae1-a24e-47cf-9c6b-147705baa36b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:55.308975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab32cc-54df-45c7-aae8-3b1be1ef7075 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:55.309090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e815ae1-a24e-47cf-9c6b-147705baa36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:55.309452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab32cc-54df-45c7-aae8-3b1be1ef7075 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:55.309916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab32cc-54df-45c7-aae8-3b1be1ef7075 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:55.311159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab32cc-54df-45c7-aae8-3b1be1ef7075 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:55.313991 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab32cc-54df-45c7-aae8-3b1be1ef7075 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:55.346367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-9e815ae1-a24e-47cf-9c6b-147705baa36b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:55.348036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9e815ae1-a24e-47cf-9c6b-147705baa36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:55.369748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:55.442981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 75/142] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:56:55 2022] POST /identity/v3/users => generated 283 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:55.475266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 68/143] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:55.880645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab32cc-54df-45c7-aae8-3b1be1ef7075 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:55.960022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:56.052603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab32cc-54df-45c7-aae8-3b1be1ef7075 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:56.100453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc8f38-0db2-4ede-9088-9c42ecc50932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:56.101131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc8f38-0db2-4ede-9088-9c42ecc50932 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:56.101420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc8f38-0db2-4ede-9088-9c42ecc50932 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:56.101704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc8f38-0db2-4ede-9088-9c42ecc50932 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:56.116465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc8f38-0db2-4ede-9088-9c42ecc50932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-3ddc8f38-0db2-4ede-9088-9c42ecc50932 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:56.122680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:56.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:56:56.125133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 76/144] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:56:56.132248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e815ae1-a24e-47cf-9c6b-147705baa36b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:56.259718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8888f5-4565-4a0d-b8f3-92f9836b96a4 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:56.260252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8888f5-4565-4a0d-b8f3-92f9836b96a4 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:56.260334 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8888f5-4565-4a0d-b8f3-92f9836b96a4 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:56.261761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 77/145] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:56:56.268518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf3e3-bdb1-4301-8d9a-387e804fb51c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:56.268815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf3e3-bdb1-4301-8d9a-387e804fb51c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:56.271248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf3e3-bdb1-4301-8d9a-387e804fb51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:56.310032 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e42cf3e3-bdb1-4301-8d9a-387e804fb51c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:56.312003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e42cf3e3-bdb1-4301-8d9a-387e804fb51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:56.323474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e815ae1-a24e-47cf-9c6b-147705baa36b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:56.391638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e56711-c5de-495a-ba46-f2983d40d76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:56.392310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56711-c5de-495a-ba46-f2983d40d76f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:56.392803 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56711-c5de-495a-ba46-f2983d40d76f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:56.393120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56711-c5de-495a-ba46-f2983d40d76f None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:56.393809 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e56711-c5de-495a-ba46-f2983d40d76f None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:56.394109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e56711-c5de-495a-ba46-f2983d40d76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:56.395526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e56711-c5de-495a-ba46-f2983d40d76f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:56.417068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 69/146] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:56:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:56.425275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e56711-c5de-495a-ba46-f2983d40d76f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:56.447412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:56.459206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 78/147] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:56.565499 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e56711-c5de-495a-ba46-f2983d40d76f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:56.616687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47535b0-960e-4cf4-b285-12ed6dafa38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:56.617911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47535b0-960e-4cf4-b285-12ed6dafa38e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:56.618208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47535b0-960e-4cf4-b285-12ed6dafa38e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:56.618478 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47535b0-960e-4cf4-b285-12ed6dafa38e None admin] PATH_INFO: `/v3/users/9d60ef5b46c84eaeaebaf462462afffb` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:56.619837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47535b0-960e-4cf4-b285-12ed6dafa38e None admin] RBAC: Authorizing `identity:get_user(user_id=9d60ef5b46c84eaeaebaf462462afffb)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:56.621431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47535b0-960e-4cf4-b285-12ed6dafa38e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:56.623624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 70/148] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:56 2022] GET /identity/v3/users/9d60ef5b46c84eaeaebaf462462afffb => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:56.632984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cf3e3-bdb1-4301-8d9a-387e804fb51c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:56.745470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:56.862028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cf3e3-bdb1-4301-8d9a-387e804fb51c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:56.924738 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47535b0-960e-4cf4-b285-12ed6dafa38e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:56.927250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046bcd8-3f19-4c1e-8800-a7d93c86888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:56.928054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046bcd8-3f19-4c1e-8800-a7d93c86888d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:56.928337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046bcd8-3f19-4c1e-8800-a7d93c86888d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:56.928615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046bcd8-3f19-4c1e-8800-a7d93c86888d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:57.039104 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:57.064600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046bcd8-3f19-4c1e-8800-a7d93c86888d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:57.066161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046bcd8-3f19-4c1e-8800-a7d93c86888d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:57.070385 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47535b0-960e-4cf4-b285-12ed6dafa38e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:57.122167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce6d91-13ec-4925-a1cb-e95b8ba248b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:57.122958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6d91-13ec-4925-a1cb-e95b8ba248b3 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:57.123366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6d91-13ec-4925-a1cb-e95b8ba248b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:57.123851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6d91-13ec-4925-a1cb-e95b8ba248b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:57.158293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:57.158917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce6d91-13ec-4925-a1cb-e95b8ba248b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b3ce6d91-13ec-4925-a1cb-e95b8ba248b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:57.165983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:57.167088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:56:57.168045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:56:57.182963 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 71/149] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:57.204502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce6d91-13ec-4925-a1cb-e95b8ba248b3 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:57.315374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 79/150] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:56:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:57.323732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046bcd8-3f19-4c1e-8800-a7d93c86888d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:57.423190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce6d91-13ec-4925-a1cb-e95b8ba248b3 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:57.535586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046bcd8-3f19-4c1e-8800-a7d93c86888d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:57.546323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85608595-004f-4724-b8ae-b5e82aa96ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:57.547048 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85608595-004f-4724-b8ae-b5e82aa96ffc None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:57.547380 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85608595-004f-4724-b8ae-b5e82aa96ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:57.547603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85608595-004f-4724-b8ae-b5e82aa96ffc None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:57.548283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85608595-004f-4724-b8ae-b5e82aa96ffc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:57.548633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85608595-004f-4724-b8ae-b5e82aa96ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:57.550567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85608595-004f-4724-b8ae-b5e82aa96ffc None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:57.569323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:57.585454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 72/151] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:56:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:57.599453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648eaaf5-bd33-4bfe-8731-7c9aebf310d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:57.600691 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eaaf5-bd33-4bfe-8731-7c9aebf310d3 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:57.601022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eaaf5-bd33-4bfe-8731-7c9aebf310d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:57.601500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eaaf5-bd33-4bfe-8731-7c9aebf310d3 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:57.602475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85608595-004f-4724-b8ae-b5e82aa96ffc None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:57.609894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648eaaf5-bd33-4bfe-8731-7c9aebf310d3 None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:57.611700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648eaaf5-bd33-4bfe-8731-7c9aebf310d3 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:57.614463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 80/152] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:56:57 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:57.621608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648eaaf5-bd33-4bfe-8731-7c9aebf310d3 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:57.646234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:57.822172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648eaaf5-bd33-4bfe-8731-7c9aebf310d3 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:57.830261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85608595-004f-4724-b8ae-b5e82aa96ffc None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:57.889211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ed9a6-803f-4d19-8de8-bb7ebae122b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:57.890216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ed9a6-803f-4d19-8de8-bb7ebae122b1 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:57.890680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ed9a6-803f-4d19-8de8-bb7ebae122b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:57.890981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ed9a6-803f-4d19-8de8-bb7ebae122b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:57.892050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ed9a6-803f-4d19-8de8-bb7ebae122b1 None admin] RBAC: Adding query filter params (role.id=13913230583d4ce384599ac5dace7ffa, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369, user.id=9d60ef5b46c84eaeaebaf462462afffb) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:57.892605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ed9a6-803f-4d19-8de8-bb7ebae122b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:57.895169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ed9a6-803f-4d19-8de8-bb7ebae122b1 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:57.914737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 81/153] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:56:57 2022] GET /identity/v3/role_assignments?user.id=9d60ef5b46c84eaeaebaf462462afffb&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=13913230583d4ce384599ac5dace7ffa => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:56:57.919319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301df125-4840-4a66-b5f7-1ec264aeff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:57.920036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301df125-4840-4a66-b5f7-1ec264aeff83 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:57.920315 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301df125-4840-4a66-b5f7-1ec264aeff83 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:57.920588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301df125-4840-4a66-b5f7-1ec264aeff83 None admin] PATH_INFO: `/v3/users/02a015006c5346118e582f84a4753f25` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:57.925876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301df125-4840-4a66-b5f7-1ec264aeff83 None admin] RBAC: Authorizing `identity:get_user(user_id=02a015006c5346118e582f84a4753f25)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:57.927522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301df125-4840-4a66-b5f7-1ec264aeff83 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:57.929316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 73/154] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:57 2022] GET /identity/v3/users/02a015006c5346118e582f84a4753f25 => generated 289 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:57.937158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ed9a6-803f-4d19-8de8-bb7ebae122b1 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:58.038658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:58.080106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ed9a6-803f-4d19-8de8-bb7ebae122b1 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:58.145901 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01eac7-3d2e-4385-ad3c-73e49c7875c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:58.146573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01eac7-3d2e-4385-ad3c-73e49c7875c3 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:58.146851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01eac7-3d2e-4385-ad3c-73e49c7875c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:58.147122 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01eac7-3d2e-4385-ad3c-73e49c7875c3 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:58.149177 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01eac7-3d2e-4385-ad3c-73e49c7875c3 None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:58.151229 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01eac7-3d2e-4385-ad3c-73e49c7875c3 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:58.153238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 82/155] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:56:57 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:58.159257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301df125-4840-4a66-b5f7-1ec264aeff83 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:58.241629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf251b-3132-4ee1-9637-216d636f5fb7 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:58.241963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf251b-3132-4ee1-9637-216d636f5fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:58.242293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf251b-3132-4ee1-9637-216d636f5fb7 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:58.244219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 83/156] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:58.256539 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad0b8c-e88c-489b-bff7-ea2bd17e15ad None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:58.257058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad0b8c-e88c-489b-bff7-ea2bd17e15ad None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:58.257542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad0b8c-e88c-489b-bff7-ea2bd17e15ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:58.301302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:58.326506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-b2ad0b8c-e88c-489b-bff7-ea2bd17e15ad None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:58.328501 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b2ad0b8c-e88c-489b-bff7-ea2bd17e15ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:58.395489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301df125-4840-4a66-b5f7-1ec264aeff83 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:58.477771 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb81595-9e3a-4b6d-a1f9-3d074b3daf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:58.478839 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb81595-9e3a-4b6d-a1f9-3d074b3daf21 None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:58.479161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb81595-9e3a-4b6d-a1f9-3d074b3daf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:58.479619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb81595-9e3a-4b6d-a1f9-3d074b3daf21 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/users/02a015006c5346118e582f84a4753f25/roles/b9b25895899d46eea0c912f6277165cb` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:58.488445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb81595-9e3a-4b6d-a1f9-3d074b3daf21 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4bdbfd07cd94eeea5087a9d39f83369, user_id=02a015006c5346118e582f84a4753f25, role_id=b9b25895899d46eea0c912f6277165cb)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:58.491043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb81595-9e3a-4b6d-a1f9-3d074b3daf21 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:58.511420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 84/157] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:56:58.580487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 74/158] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:56:58 2022] PUT /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/users/02a015006c5346118e582f84a4753f25/roles/b9b25895899d46eea0c912f6277165cb => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:56:58.945680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb9ded-8184-4485-be34-d1fd4cef73c8 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:58.946181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb9ded-8184-4485-be34-d1fd4cef73c8 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:58.946367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb9ded-8184-4485-be34-d1fd4cef73c8 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:58.948629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 85/159] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:58.955177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bd9a1-fb92-4dc3-95be-868ccb0563bd None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:58.955567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bd9a1-fb92-4dc3-95be-868ccb0563bd None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:58.955824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bd9a1-fb92-4dc3-95be-868ccb0563bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:58.980168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-6b3bd9a1-fb92-4dc3-95be-868ccb0563bd None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:58.981804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6b3bd9a1-fb92-4dc3-95be-868ccb0563bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:59.071271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 75/160] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:56:59.096841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb9ded-8184-4485-be34-d1fd4cef73c8 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:59.211154 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:59.253098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb9ded-8184-4485-be34-d1fd4cef73c8 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:59.310966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9b8ab-d95d-4238-bc55-3f6a56cc6a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:59.312307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b8ab-d95d-4238-bc55-3f6a56cc6a69 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:59.312825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b8ab-d95d-4238-bc55-3f6a56cc6a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:59.313328 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b8ab-d95d-4238-bc55-3f6a56cc6a69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:59.330292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9b8ab-d95d-4238-bc55-3f6a56cc6a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-def9b8ab-d95d-4238-bc55-3f6a56cc6a69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:56:59.336575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:56:59.337579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:56:59.339375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:56:59.345295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 86/161] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:56:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:56:59.351969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bd9a1-fb92-4dc3-95be-868ccb0563bd None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:59.411942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379a949-c348-4eaa-8cba-9f09afab247f None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:59.412831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379a949-c348-4eaa-8cba-9f09afab247f None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:59.413346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379a949-c348-4eaa-8cba-9f09afab247f None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:59.416243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 87/162] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:56:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:56:59.427120 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ca67e-20d9-4654-aec5-4a517d40ad57 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:59.427723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ca67e-20d9-4654-aec5-4a517d40ad57 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:59.428203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ca67e-20d9-4654-aec5-4a517d40ad57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:59.460579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c96ca67e-20d9-4654-aec5-4a517d40ad57 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:59.463118 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c96ca67e-20d9-4654-aec5-4a517d40ad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:59.481131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:59.574036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bd9a1-fb92-4dc3-95be-868ccb0563bd None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:59.593008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 88/163] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:56:59.637649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680aef40-b490-4445-935b-05d42283e618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:59.638280 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aef40-b490-4445-935b-05d42283e618 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:59.638559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aef40-b490-4445-935b-05d42283e618 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:59.638838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aef40-b490-4445-935b-05d42283e618 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:59.639518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680aef40-b490-4445-935b-05d42283e618 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:59.639820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680aef40-b490-4445-935b-05d42283e618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:59.641242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680aef40-b490-4445-935b-05d42283e618 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:59.652128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 76/164] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:59.660615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ca67e-20d9-4654-aec5-4a517d40ad57 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:59.694050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680aef40-b490-4445-935b-05d42283e618 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:59.770558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:59.899105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ca67e-20d9-4654-aec5-4a517d40ad57 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:59.932155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680aef40-b490-4445-935b-05d42283e618 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:59.979463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:59.985790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8500e62-3388-4d0c-a6ac-36266488b1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:59.986739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8500e62-3388-4d0c-a6ac-36266488b1fa None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:59.987222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8500e62-3388-4d0c-a6ac-36266488b1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:59.989404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8500e62-3388-4d0c-a6ac-36266488b1fa None admin] PATH_INFO: `/v3/users/deb4aa9488e54d0eafc940ddd57f345d` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:59.990871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8500e62-3388-4d0c-a6ac-36266488b1fa None admin] RBAC: Authorizing `identity:get_user(user_id=deb4aa9488e54d0eafc940ddd57f345d)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:59.995635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8500e62-3388-4d0c-a6ac-36266488b1fa None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:00.001629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 89/165] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:56:59 2022] GET /identity/v3/users/deb4aa9488e54d0eafc940ddd57f345d => generated 247 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:00.004781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8500e62-3388-4d0c-a6ac-36266488b1fa None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:00.025762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62904895-8054-4561-b365-a7f4a2c2505d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:00.026782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62904895-8054-4561-b365-a7f4a2c2505d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:00.027321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62904895-8054-4561-b365-a7f4a2c2505d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:00.027796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62904895-8054-4561-b365-a7f4a2c2505d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:00.046117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62904895-8054-4561-b365-a7f4a2c2505d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-62904895-8054-4561-b365-a7f4a2c2505d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:00.052645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:00.053685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:00.054475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:00.062887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 77/166] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:56:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:57:00.069464 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62904895-8054-4561-b365-a7f4a2c2505d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:00.204064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62904895-8054-4561-b365-a7f4a2c2505d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:00.247630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4175ed0-d2f6-4071-9254-be47200fa3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:00.248260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4175ed0-d2f6-4071-9254-be47200fa3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:00.248539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4175ed0-d2f6-4071-9254-be47200fa3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:00.248817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4175ed0-d2f6-4071-9254-be47200fa3d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:00.249525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4175ed0-d2f6-4071-9254-be47200fa3d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:00.249828 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4175ed0-d2f6-4071-9254-be47200fa3d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:00.251216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4175ed0-d2f6-4071-9254-be47200fa3d4 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:00.262302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8500e62-3388-4d0c-a6ac-36266488b1fa None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:00.271991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 78/167] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:00.283903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4175ed0-d2f6-4071-9254-be47200fa3d4 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:00.333954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:00.355768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d1ebb-432f-4eef-b63b-0a5cd22547ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:00.356392 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d1ebb-432f-4eef-b63b-0a5cd22547ad None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:00.356678 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d1ebb-432f-4eef-b63b-0a5cd22547ad None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:00.356987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d1ebb-432f-4eef-b63b-0a5cd22547ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:00.396601 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:00.490396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4175ed0-d2f6-4071-9254-be47200fa3d4 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:00.509255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d1ebb-432f-4eef-b63b-0a5cd22547ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:00.511077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d1ebb-432f-4eef-b63b-0a5cd22547ad None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:00.539922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfc93b-87f0-419d-998d-8431c4997626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:00.541164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfc93b-87f0-419d-998d-8431c4997626 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:00.541651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfc93b-87f0-419d-998d-8431c4997626 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:00.542757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfc93b-87f0-419d-998d-8431c4997626 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:00.564626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfc93b-87f0-419d-998d-8431c4997626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-7fcfc93b-87f0-419d-998d-8431c4997626 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:00.572386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:00.573939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:00.575135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:00.582556 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 79/168] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:57:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:00.585451 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfc93b-87f0-419d-998d-8431c4997626 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:00.699071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 90/169] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:00.701060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f91cf7-b568-4da1-9e51-782c8ae57ef5 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:00.701558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f91cf7-b568-4da1-9e51-782c8ae57ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:00.702034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f91cf7-b568-4da1-9e51-782c8ae57ef5 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:00.703567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 91/170] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:00.709054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f91cf7-b568-4da1-9e51-782c8ae57ef5 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:00.757265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:00.834891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f91cf7-b568-4da1-9e51-782c8ae57ef5 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:00.866674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfc93b-87f0-419d-998d-8431c4997626 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:00.869287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:00.878025 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c243cf-8de0-4063-ba06-24aba62c25af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:00.879120 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c243cf-8de0-4063-ba06-24aba62c25af None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:00.881980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c243cf-8de0-4063-ba06-24aba62c25af None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:00.881980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c243cf-8de0-4063-ba06-24aba62c25af None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:00.881980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c243cf-8de0-4063-ba06-24aba62c25af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:00.881980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c243cf-8de0-4063-ba06-24aba62c25af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:00.883708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c243cf-8de0-4063-ba06-24aba62c25af None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:00.893576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 92/171] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:00.894795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c243cf-8de0-4063-ba06-24aba62c25af None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:00.972291 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac08c7-13d5-4f3b-b5c7-e8b8172cba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:00.972291 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac08c7-13d5-4f3b-b5c7-e8b8172cba1a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:00.972291 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac08c7-13d5-4f3b-b5c7-e8b8172cba1a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:00.972291 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac08c7-13d5-4f3b-b5c7-e8b8172cba1a None admin] PATH_INFO: `/v3/users/9d60ef5b46c84eaeaebaf462462afffb` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:00.972790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac08c7-13d5-4f3b-b5c7-e8b8172cba1a None admin] RBAC: Authorizing `identity:get_user(user_id=9d60ef5b46c84eaeaebaf462462afffb)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:00.972790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac08c7-13d5-4f3b-b5c7-e8b8172cba1a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:00.972790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 80/172] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:00 2022] GET /identity/v3/users/9d60ef5b46c84eaeaebaf462462afffb => generated 275 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:00.972790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77ddc2-d6b6-4659-975e-9ef877ed93bc None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:00.973326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77ddc2-d6b6-4659-975e-9ef877ed93bc None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:00.973415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77ddc2-d6b6-4659-975e-9ef877ed93bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.016772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-4e77ddc2-d6b6-4659-975e-9ef877ed93bc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:01.018195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4e77ddc2-d6b6-4659-975e-9ef877ed93bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:01.028023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c243cf-8de0-4063-ba06-24aba62c25af None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:01.041345 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:01.069364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4cd9a-020c-466a-947b-30806ef5ba20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:01.070894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4cd9a-020c-466a-947b-30806ef5ba20 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.071371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4cd9a-020c-466a-947b-30806ef5ba20 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.071825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4cd9a-020c-466a-947b-30806ef5ba20 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.073237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4cd9a-020c-466a-947b-30806ef5ba20 None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:01.074899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4cd9a-020c-466a-947b-30806ef5ba20 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:01.077031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 93/173] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:00 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:01.078140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4cd9a-020c-466a-947b-30806ef5ba20 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:01.129754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 81/174] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:01.133760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77ddc2-d6b6-4659-975e-9ef877ed93bc None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:01.209981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4cd9a-020c-466a-947b-30806ef5ba20 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:01.252502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6c121-0831-441c-9364-f35001bc991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:01.253650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c121-0831-441c-9364-f35001bc991a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.254382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c121-0831-441c-9364-f35001bc991a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.254833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c121-0831-441c-9364-f35001bc991a None admin] PATH_INFO: `/v3/users/9d60ef5b46c84eaeaebaf462462afffb` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.256466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6c121-0831-441c-9364-f35001bc991a None admin] RBAC: Authorizing `identity:get_user(user_id=9d60ef5b46c84eaeaebaf462462afffb)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:01.258684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6c121-0831-441c-9364-f35001bc991a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:01.260657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 94/175] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:01 2022] GET /identity/v3/users/9d60ef5b46c84eaeaebaf462462afffb => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:01.262428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b35ab7-161f-410e-80fc-ed36981e0b70 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.262899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b35ab7-161f-410e-80fc-ed36981e0b70 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.263319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b35ab7-161f-410e-80fc-ed36981e0b70 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.265586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 95/176] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:01.266904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e61297-d73e-48c8-8a54-78f429262821 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.267387 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e61297-d73e-48c8-8a54-78f429262821 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.267812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e61297-d73e-48c8-8a54-78f429262821 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.269235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 96/177] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:01.270093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e61297-d73e-48c8-8a54-78f429262821 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:01.279554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:01.283736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77ddc2-d6b6-4659-975e-9ef877ed93bc None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:01.309387 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:01.333245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5d069-e03a-462d-aedf-cef3708a18c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:01.334335 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d069-e03a-462d-aedf-cef3708a18c8 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.334716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d069-e03a-462d-aedf-cef3708a18c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.335089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d069-e03a-462d-aedf-cef3708a18c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.398395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e61297-d73e-48c8-8a54-78f429262821 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:01.428423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5d069-e03a-462d-aedf-cef3708a18c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:01.430999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5d069-e03a-462d-aedf-cef3708a18c8 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:01.446971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905381b-ffd1-4511-a9c6-6af338dbd0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:01.447612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905381b-ffd1-4511-a9c6-6af338dbd0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.447897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905381b-ffd1-4511-a9c6-6af338dbd0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.448176 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905381b-ffd1-4511-a9c6-6af338dbd0f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.448963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905381b-ffd1-4511-a9c6-6af338dbd0f5 None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369, user.id=deb4aa9488e54d0eafc940ddd57f345d) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:01.449270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905381b-ffd1-4511-a9c6-6af338dbd0f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:01.450710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905381b-ffd1-4511-a9c6-6af338dbd0f5 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:01.456692 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 97/178] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:01 2022] GET /identity/v3/role_assignments?user.id=deb4aa9488e54d0eafc940ddd57f345d&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:01.457528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905381b-ffd1-4511-a9c6-6af338dbd0f5 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:01.568923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:01.585399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:01.588743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905381b-ffd1-4511-a9c6-6af338dbd0f5 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:01.596079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 82/179] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:01.598386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a649dd-30d3-4657-afb4-54ff1fd69071 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.598670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a649dd-30d3-4657-afb4-54ff1fd69071 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.598944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a649dd-30d3-4657-afb4-54ff1fd69071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.623337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-41a649dd-30d3-4657-afb4-54ff1fd69071 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:01.625027 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-41a649dd-30d3-4657-afb4-54ff1fd69071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:01.639774 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb93ec-8726-4ead-bb24-7d3962465eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:01.640700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb93ec-8726-4ead-bb24-7d3962465eec None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.641047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb93ec-8726-4ead-bb24-7d3962465eec None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.641331 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb93ec-8726-4ead-bb24-7d3962465eec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.721928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 83/180] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:01.723551 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f9fdc-96f7-49fd-8bd4-1e08cc2b8be8 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.723551 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f9fdc-96f7-49fd-8bd4-1e08cc2b8be8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.723860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f9fdc-96f7-49fd-8bd4-1e08cc2b8be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.748461 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-8f2f9fdc-96f7-49fd-8bd4-1e08cc2b8be8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:01.750093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8f2f9fdc-96f7-49fd-8bd4-1e08cc2b8be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:01.752447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb93ec-8726-4ead-bb24-7d3962465eec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:01.754365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb93ec-8726-4ead-bb24-7d3962465eec None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:01.823122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 84/181] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:01.833866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f9fdc-96f7-49fd-8bd4-1e08cc2b8be8 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:01.861951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:01.878043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 98/182] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:01.887951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb93ec-8726-4ead-bb24-7d3962465eec None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:01.905163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:01.955196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f9fdc-96f7-49fd-8bd4-1e08cc2b8be8 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:01.994042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffe103-3796-4496-9d6a-d6ed3de8654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:01.994252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb93ec-8726-4ead-bb24-7d3962465eec None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:01.995199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffe103-3796-4496-9d6a-d6ed3de8654a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.995848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffe103-3796-4496-9d6a-d6ed3de8654a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.996346 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffe103-3796-4496-9d6a-d6ed3de8654a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.011056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffe103-3796-4496-9d6a-d6ed3de8654a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-54ffe103-3796-4496-9d6a-d6ed3de8654a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:02.020716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:02.021827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:02.022901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 85/183] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:57:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:02.024347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5818258-8b9b-4dd0-a64e-d4b8bb6fe427 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.024347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5818258-8b9b-4dd0-a64e-d4b8bb6fe427 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.024347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5818258-8b9b-4dd0-a64e-d4b8bb6fe427 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.025691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 86/184] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:02.026252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5818258-8b9b-4dd0-a64e-d4b8bb6fe427 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:02.042042 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8e072-96b4-45bd-8d79-452c1172ce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:02.042681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8e072-96b4-45bd-8d79-452c1172ce61 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.042966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8e072-96b4-45bd-8d79-452c1172ce61 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.043244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8e072-96b4-45bd-8d79-452c1172ce61 None admin] PATH_INFO: `/v3/projects/e73480bacc274d19b1b2bb39dd93bc9b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.044418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8e072-96b4-45bd-8d79-452c1172ce61 None admin] RBAC: Authorizing `identity:get_project(project_id=e73480bacc274d19b1b2bb39dd93bc9b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:02.045904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8e072-96b4-45bd-8d79-452c1172ce61 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:02.047759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 99/185] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:01 2022] GET /identity/v3/projects/e73480bacc274d19b1b2bb39dd93bc9b => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:02.053138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8e072-96b4-45bd-8d79-452c1172ce61 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:02.118592 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:02.133914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5818258-8b9b-4dd0-a64e-d4b8bb6fe427 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:02.154590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:02.160537 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8e072-96b4-45bd-8d79-452c1172ce61 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:02.167620 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec658c-ce99-44ad-9305-44112d603436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:02.168246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec658c-ce99-44ad-9305-44112d603436 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.168520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec658c-ce99-44ad-9305-44112d603436 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.168790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec658c-ce99-44ad-9305-44112d603436 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.169941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec658c-ce99-44ad-9305-44112d603436 None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:02.171449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec658c-ce99-44ad-9305-44112d603436 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:02.173085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 87/186] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:02 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:02.174182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4927a6c-36db-4105-a9ac-87c4849006fe None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.174458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4927a6c-36db-4105-a9ac-87c4849006fe None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.174726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4927a6c-36db-4105-a9ac-87c4849006fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.193032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b4927a6c-36db-4105-a9ac-87c4849006fe None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:02.194254 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b4927a6c-36db-4105-a9ac-87c4849006fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:02.198911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfd781-caa4-4913-a8c6-fbd9c2975742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:02.199540 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfd781-caa4-4913-a8c6-fbd9c2975742 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.199822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfd781-caa4-4913-a8c6-fbd9c2975742 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.200183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfd781-caa4-4913-a8c6-fbd9c2975742 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.201322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfd781-caa4-4913-a8c6-fbd9c2975742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:02.226683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfd781-caa4-4913-a8c6-fbd9c2975742 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:02.226683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 100/187] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:02.226683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfd781-caa4-4913-a8c6-fbd9c2975742 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:02.265172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 88/188] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:02.266076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4927a6c-36db-4105-a9ac-87c4849006fe None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:02.321664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfd781-caa4-4913-a8c6-fbd9c2975742 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:02.350365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:02.360183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c243a-6666-46ce-bddb-b2049398e68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:02.360800 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c243a-6666-46ce-bddb-b2049398e68a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.361111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c243a-6666-46ce-bddb-b2049398e68a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.361386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c243a-6666-46ce-bddb-b2049398e68a None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.362050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c243a-6666-46ce-bddb-b2049398e68a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:02.362352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c243a-6666-46ce-bddb-b2049398e68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:02.363735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c243a-6666-46ce-bddb-b2049398e68a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:02.373276 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 101/189] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:02.374165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c243a-6666-46ce-bddb-b2049398e68a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:02.392159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4927a6c-36db-4105-a9ac-87c4849006fe None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:02.411820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:02.430524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7942a-d461-4898-bff1-d53ce4831c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:02.431547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7942a-d461-4898-bff1-d53ce4831c17 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.431835 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7942a-d461-4898-bff1-d53ce4831c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.432104 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7942a-d461-4898-bff1-d53ce4831c17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.432844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7942a-d461-4898-bff1-d53ce4831c17 None admin] RBAC: Adding query filter params (role.id=13913230583d4ce384599ac5dace7ffa, scope.project.id=e73480bacc274d19b1b2bb39dd93bc9b, user.id=9d60ef5b46c84eaeaebaf462462afffb) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:02.433181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7942a-d461-4898-bff1-d53ce4831c17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:02.435770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7942a-d461-4898-bff1-d53ce4831c17 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:02.452898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 89/190] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:02 2022] GET /identity/v3/role_assignments?user.id=9d60ef5b46c84eaeaebaf462462afffb&scope.project.id=e73480bacc274d19b1b2bb39dd93bc9b&role.id=13913230583d4ce384599ac5dace7ffa => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:02.452898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7942a-d461-4898-bff1-d53ce4831c17 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:02.494509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c243a-6666-46ce-bddb-b2049398e68a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:02.545044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fec8ae-8dab-4b41-bc12-3d39ded3cbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:02.545662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fec8ae-8dab-4b41-bc12-3d39ded3cbd6 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.545946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fec8ae-8dab-4b41-bc12-3d39ded3cbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.546217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fec8ae-8dab-4b41-bc12-3d39ded3cbd6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.560253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fec8ae-8dab-4b41-bc12-3d39ded3cbd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-15fec8ae-8dab-4b41-bc12-3d39ded3cbd6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:02.565865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:02.566816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 102/191] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:57:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:02.569316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fec8ae-8dab-4b41-bc12-3d39ded3cbd6 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:02.602103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7942a-d461-4898-bff1-d53ce4831c17 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:02.625434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:02.648584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f845b-b666-4267-b116-6a6086ae1add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:02.649104 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f845b-b666-4267-b116-6a6086ae1add None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.649382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f845b-b666-4267-b116-6a6086ae1add None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.649648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f845b-b666-4267-b116-6a6086ae1add None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.650386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f845b-b666-4267-b116-6a6086ae1add None admin] RBAC: Adding query filter params (role.id=276ae07995d14b5d8ceb8e646392c2b1, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369, user.id=9d60ef5b46c84eaeaebaf462462afffb) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:02.650689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f845b-b666-4267-b116-6a6086ae1add None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:02.652342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f845b-b666-4267-b116-6a6086ae1add None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:02.658918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 90/192] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:02 2022] GET /identity/v3/role_assignments?user.id=9d60ef5b46c84eaeaebaf462462afffb&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=276ae07995d14b5d8ceb8e646392c2b1 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:02.659829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f845b-b666-4267-b116-6a6086ae1add None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:02.688632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fec8ae-8dab-4b41-bc12-3d39ded3cbd6 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:02.722566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:02.741763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2929fa3-0517-4869-bf93-f5cacfda0839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:02.743348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2929fa3-0517-4869-bf93-f5cacfda0839 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.743698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2929fa3-0517-4869-bf93-f5cacfda0839 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.744010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2929fa3-0517-4869-bf93-f5cacfda0839 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.751475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2929fa3-0517-4869-bf93-f5cacfda0839 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:02.751475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2929fa3-0517-4869-bf93-f5cacfda0839 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:02.781974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f845b-b666-4267-b116-6a6086ae1add None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:02.818933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f4d39-3933-441f-baf9-f5627550f9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:02.819505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f4d39-3933-441f-baf9-f5627550f9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.819792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f4d39-3933-441f-baf9-f5627550f9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.820072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f4d39-3933-441f-baf9-f5627550f9d7 None admin] PATH_INFO: `/v3/users/47aad92357db4357b5b7b484dc866813` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.834914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f4d39-3933-441f-baf9-f5627550f9d7 None admin] RBAC: Authorizing `identity:get_user(user_id=47aad92357db4357b5b7b484dc866813)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:02.836826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f4d39-3933-441f-baf9-f5627550f9d7 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:02.839527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 91/193] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:02 2022] GET /identity/v3/users/47aad92357db4357b5b7b484dc866813 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:02.840331 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f4d39-3933-441f-baf9-f5627550f9d7 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:02.888179 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 103/194] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:57:02 2022] POST /identity/v3/users => generated 297 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:02.889363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2929fa3-0517-4869-bf93-f5cacfda0839 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:02.967273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f4d39-3933-441f-baf9-f5627550f9d7 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:02.972773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:03.011065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2929fa3-0517-4869-bf93-f5cacfda0839 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:03.014004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:03.020354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2443afc-5cf7-4868-b993-819f5696f08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:03.021048 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2443afc-5cf7-4868-b993-819f5696f08a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:03.021337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2443afc-5cf7-4868-b993-819f5696f08a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:03.021614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2443afc-5cf7-4868-b993-819f5696f08a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:03.034778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2443afc-5cf7-4868-b993-819f5696f08a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b2443afc-5cf7-4868-b993-819f5696f08a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:03.040804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:03.046379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 92/195] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:03.047185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2443afc-5cf7-4868-b993-819f5696f08a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:03.062097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642632f8-0ec3-4e19-b9a4-39e90674ca6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:03.062726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642632f8-0ec3-4e19-b9a4-39e90674ca6f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:03.063010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642632f8-0ec3-4e19-b9a4-39e90674ca6f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:03.063372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642632f8-0ec3-4e19-b9a4-39e90674ca6f None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:03.064025 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642632f8-0ec3-4e19-b9a4-39e90674ca6f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:03.064334 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642632f8-0ec3-4e19-b9a4-39e90674ca6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:03.065854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642632f8-0ec3-4e19-b9a4-39e90674ca6f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:03.076289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 104/196] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:03.077386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642632f8-0ec3-4e19-b9a4-39e90674ca6f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:03.181910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2443afc-5cf7-4868-b993-819f5696f08a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:03.206881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642632f8-0ec3-4e19-b9a4-39e90674ca6f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:03.229160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ab7bb-2e50-4977-a9fb-80e2bd1b8a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:03.229789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ab7bb-2e50-4977-a9fb-80e2bd1b8a6a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:03.230069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ab7bb-2e50-4977-a9fb-80e2bd1b8a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:03.230347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ab7bb-2e50-4977-a9fb-80e2bd1b8a6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:03.252617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89271592-490b-44d7-af9f-5256f5b6c047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:03.253636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89271592-490b-44d7-af9f-5256f5b6c047 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:03.254050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89271592-490b-44d7-af9f-5256f5b6c047 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:03.254493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89271592-490b-44d7-af9f-5256f5b6c047 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:03.255176 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89271592-490b-44d7-af9f-5256f5b6c047 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:03.255644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89271592-490b-44d7-af9f-5256f5b6c047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:03.257381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89271592-490b-44d7-af9f-5256f5b6c047 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:03.268332 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 105/197] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:03.270156 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89271592-490b-44d7-af9f-5256f5b6c047 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:03.285052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:03.316008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:03.339913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ab7bb-2e50-4977-a9fb-80e2bd1b8a6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:03.341356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ab7bb-2e50-4977-a9fb-80e2bd1b8a6a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:03.414642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89271592-490b-44d7-af9f-5256f5b6c047 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:03.453285 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4de12c-325c-42b9-a2a3-5cc4dc36a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:03.454390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4de12c-325c-42b9-a2a3-5cc4dc36a186 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:03.454851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4de12c-325c-42b9-a2a3-5cc4dc36a186 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:03.455300 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4de12c-325c-42b9-a2a3-5cc4dc36a186 None admin] PATH_INFO: `/v3/users/47aad92357db4357b5b7b484dc866813` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:03.456622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4de12c-325c-42b9-a2a3-5cc4dc36a186 None admin] RBAC: Authorizing `identity:get_user(user_id=47aad92357db4357b5b7b484dc866813)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:03.458142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 93/198] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:03.458973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4de12c-325c-42b9-a2a3-5cc4dc36a186 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:03.459412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ab7bb-2e50-4977-a9fb-80e2bd1b8a6a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:03.461909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 106/199] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:03 2022] GET /identity/v3/users/47aad92357db4357b5b7b484dc866813 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:03.472827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4de12c-325c-42b9-a2a3-5cc4dc36a186 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:03.532057 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:03.575369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ab7bb-2e50-4977-a9fb-80e2bd1b8a6a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:03.599597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:03.607903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4de12c-325c-42b9-a2a3-5cc4dc36a186 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:03.617217 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9570c-644c-433b-a759-4cd42b44240a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:03.618135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9570c-644c-433b-a759-4cd42b44240a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:03.620894 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9570c-644c-433b-a759-4cd42b44240a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:03.620894 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9570c-644c-433b-a759-4cd42b44240a None admin] PATH_INFO: `/v3/users/02a015006c5346118e582f84a4753f25` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:03.620894 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9570c-644c-433b-a759-4cd42b44240a None admin] RBAC: Authorizing `identity:get_user(user_id=02a015006c5346118e582f84a4753f25)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:03.621388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9570c-644c-433b-a759-4cd42b44240a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:03.622975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 94/200] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:03 2022] GET /identity/v3/users/02a015006c5346118e582f84a4753f25 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:03.623818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9570c-644c-433b-a759-4cd42b44240a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:03.653487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beca6ef-8533-48bc-bce3-c0f708d6ffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:03.654363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beca6ef-8533-48bc-bce3-c0f708d6ffc8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:03.654654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beca6ef-8533-48bc-bce3-c0f708d6ffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:03.654959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beca6ef-8533-48bc-bce3-c0f708d6ffc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:03.734799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9570c-644c-433b-a759-4cd42b44240a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:03.749992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beca6ef-8533-48bc-bce3-c0f708d6ffc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:03.752180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beca6ef-8533-48bc-bce3-c0f708d6ffc8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:03.769466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:03.775775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3044a-5f7d-4307-b2e5-21db2df02c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:03.776743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3044a-5f7d-4307-b2e5-21db2df02c96 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:03.777338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3044a-5f7d-4307-b2e5-21db2df02c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:03.777829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3044a-5f7d-4307-b2e5-21db2df02c96 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:03.779170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf3044a-5f7d-4307-b2e5-21db2df02c96 None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:03.780767 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf3044a-5f7d-4307-b2e5-21db2df02c96 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:03.782669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 95/201] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:03 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:03.784631 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3044a-5f7d-4307-b2e5-21db2df02c96 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:03.887055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:03.901142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3044a-5f7d-4307-b2e5-21db2df02c96 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:03.901666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 107/202] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:03.902991 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beca6ef-8533-48bc-bce3-c0f708d6ffc8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:03.938093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb0053-06f6-4c0c-9e39-938e227760bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:03.938698 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0053-06f6-4c0c-9e39-938e227760bc None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:03.938974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0053-06f6-4c0c-9e39-938e227760bc None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:03.939242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0053-06f6-4c0c-9e39-938e227760bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:04.011645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:04.014802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb0053-06f6-4c0c-9e39-938e227760bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:04.015602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beca6ef-8533-48bc-bce3-c0f708d6ffc8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:04.016259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb0053-06f6-4c0c-9e39-938e227760bc None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:04.054042 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cd238-0229-46aa-81b4-217f05604082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:04.055545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd238-0229-46aa-81b4-217f05604082 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:04.056032 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd238-0229-46aa-81b4-217f05604082 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:04.056540 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd238-0229-46aa-81b4-217f05604082 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:04.057526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001cd238-0229-46aa-81b4-217f05604082 None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=42ce709191ce4b1b961e89e218566b5b, user.id=47aad92357db4357b5b7b484dc866813) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:04.058134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001cd238-0229-46aa-81b4-217f05604082 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:04.059852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001cd238-0229-46aa-81b4-217f05604082 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:04.066153 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 108/203] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:03 2022] GET /identity/v3/role_assignments?user.id=47aad92357db4357b5b7b484dc866813&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:04.074117 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cd238-0229-46aa-81b4-217f05604082 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:04.133421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 96/204] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:04.134820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb0053-06f6-4c0c-9e39-938e227760bc None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:04.193225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:04.247164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:04.252955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb0053-06f6-4c0c-9e39-938e227760bc None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:04.264436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cd238-0229-46aa-81b4-217f05604082 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:04.302204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88f7d4-a68b-4d2d-bc5c-58319d341b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:04.302988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88f7d4-a68b-4d2d-bc5c-58319d341b5f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:04.303355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88f7d4-a68b-4d2d-bc5c-58319d341b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:04.303736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88f7d4-a68b-4d2d-bc5c-58319d341b5f None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:04.305772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88f7d4-a68b-4d2d-bc5c-58319d341b5f None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:04.307596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88f7d4-a68b-4d2d-bc5c-58319d341b5f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:04.310055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 97/205] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:04 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:04.316230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88f7d4-a68b-4d2d-bc5c-58319d341b5f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:04.366139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc462468-3cdc-4681-8f85-d8a055f1adbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:04.366790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc462468-3cdc-4681-8f85-d8a055f1adbc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:04.367687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc462468-3cdc-4681-8f85-d8a055f1adbc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:04.367976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc462468-3cdc-4681-8f85-d8a055f1adbc None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:04.373597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc462468-3cdc-4681-8f85-d8a055f1adbc None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:04.375097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc462468-3cdc-4681-8f85-d8a055f1adbc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:04.376793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 109/206] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:04 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:04.382960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc462468-3cdc-4681-8f85-d8a055f1adbc None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:04.440946 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88f7d4-a68b-4d2d-bc5c-58319d341b5f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:04.484468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1467d-f269-4409-82c7-b02ce131060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:04.485335 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1467d-f269-4409-82c7-b02ce131060a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:04.485618 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1467d-f269-4409-82c7-b02ce131060a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:04.485885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1467d-f269-4409-82c7-b02ce131060a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:04.486621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1467d-f269-4409-82c7-b02ce131060a None admin] RBAC: Adding query filter params (role.id=b9b25895899d46eea0c912f6277165cb, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369, user.id=02a015006c5346118e582f84a4753f25) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:04.486922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1467d-f269-4409-82c7-b02ce131060a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:04.488384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1467d-f269-4409-82c7-b02ce131060a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:04.495927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 98/207] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:04 2022] GET /identity/v3/role_assignments?user.id=02a015006c5346118e582f84a4753f25&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=b9b25895899d46eea0c912f6277165cb => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:04.566046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc462468-3cdc-4681-8f85-d8a055f1adbc None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:04.596217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:04.604745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947772d1-0e3c-45c3-87d0-a76030623b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:04.605555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947772d1-0e3c-45c3-87d0-a76030623b35 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:04.605711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947772d1-0e3c-45c3-87d0-a76030623b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:04.605905 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947772d1-0e3c-45c3-87d0-a76030623b35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:04.606641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947772d1-0e3c-45c3-87d0-a76030623b35 None admin] RBAC: Adding query filter params (role.id=276ae07995d14b5d8ceb8e646392c2b1, scope.project.id=42ce709191ce4b1b961e89e218566b5b, user.id=47aad92357db4357b5b7b484dc866813) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:04.606947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947772d1-0e3c-45c3-87d0-a76030623b35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:04.608395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947772d1-0e3c-45c3-87d0-a76030623b35 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:04.615381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 110/208] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:04 2022] GET /identity/v3/role_assignments?user.id=47aad92357db4357b5b7b484dc866813&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=276ae07995d14b5d8ceb8e646392c2b1 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:05.085546 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850c5c2-4f39-4769-9906-7db08e9d8cb8 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:05.086239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850c5c2-4f39-4769-9906-7db08e9d8cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:05.086239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850c5c2-4f39-4769-9906-7db08e9d8cb8 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:05.087504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 99/209] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:05.093284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b3d1c-58de-4727-bda0-db726e7d1c4b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:05.093578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b3d1c-58de-4727-bda0-db726e7d1c4b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:05.093849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b3d1c-58de-4727-bda0-db726e7d1c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:05.115563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-541b3d1c-58de-4727-bda0-db726e7d1c4b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:05.116786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-541b3d1c-58de-4727-bda0-db726e7d1c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:05.203255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 111/210] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:05.319843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da51ac-8a20-4081-9b05-b244a337fc06 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:05.320436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da51ac-8a20-4081-9b05-b244a337fc06 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:05.321083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da51ac-8a20-4081-9b05-b244a337fc06 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:05.323102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 100/211] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:05.331723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc393d-2b22-452a-beea-f71f5ae96ef2 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:05.331723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc393d-2b22-452a-beea-f71f5ae96ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:05.331723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc393d-2b22-452a-beea-f71f5ae96ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:05.451845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-fabc393d-2b22-452a-beea-f71f5ae96ef2 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:05.453532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-fabc393d-2b22-452a-beea-f71f5ae96ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:05.490117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da51ac-8a20-4081-9b05-b244a337fc06 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:05.540834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:05.544752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 112/212] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:05.623000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da51ac-8a20-4081-9b05-b244a337fc06 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:05.659642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5077fe-5fd5-4753-8235-37a111f01a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:05.660636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5077fe-5fd5-4753-8235-37a111f01a20 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:05.661147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5077fe-5fd5-4753-8235-37a111f01a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:05.661621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5077fe-5fd5-4753-8235-37a111f01a20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:05.674321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5077fe-5fd5-4753-8235-37a111f01a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-9c5077fe-5fd5-4753-8235-37a111f01a20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:05.680016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:05.681000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:05.681818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:05.684365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 101/213] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:05.690471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc393d-2b22-452a-beea-f71f5ae96ef2 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:05.750077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:05.801970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc393d-2b22-452a-beea-f71f5ae96ef2 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:05.831871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5077fe-5fd5-4753-8235-37a111f01a20 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:05.843019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9b9ff-3975-4dd5-88f3-f6bea99f7f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:05.843660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9b9ff-3975-4dd5-88f3-f6bea99f7f0d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:05.843943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9b9ff-3975-4dd5-88f3-f6bea99f7f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:05.844224 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9b9ff-3975-4dd5-88f3-f6bea99f7f0d None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:05.844928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9b9ff-3975-4dd5-88f3-f6bea99f7f0d None admin] RBAC: Adding query filter params (name=member) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:05.845242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9b9ff-3975-4dd5-88f3-f6bea99f7f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:05.846795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9b9ff-3975-4dd5-88f3-f6bea99f7f0d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:05.856451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 113/214] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:05.862083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9b9ff-3975-4dd5-88f3-f6bea99f7f0d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:05.956930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:05.967433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5077fe-5fd5-4753-8235-37a111f01a20 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:05.986009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9b9ff-3975-4dd5-88f3-f6bea99f7f0d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:06.002472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:06.005453 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73d391-4180-4978-8600-c276d20c7c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:06.006447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d391-4180-4978-8600-c276d20c7c50 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.007090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d391-4180-4978-8600-c276d20c7c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.007617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d391-4180-4978-8600-c276d20c7c50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.009056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73d391-4180-4978-8600-c276d20c7c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:06.010779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73d391-4180-4978-8600-c276d20c7c50 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:06.012639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 102/215] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:06.018410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73d391-4180-4978-8600-c276d20c7c50 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:06.035058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb6a1d-466b-4725-836f-c1c5534449a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:06.036108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6a1d-466b-4725-836f-c1c5534449a4 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.036471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6a1d-466b-4725-836f-c1c5534449a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.036839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6a1d-466b-4725-836f-c1c5534449a4 None admin] PATH_INFO: `/v3/users/b744e4892be341558b8f28271ca52907` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.056993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb6a1d-466b-4725-836f-c1c5534449a4 None admin] RBAC: Authorizing `identity:get_user(user_id=b744e4892be341558b8f28271ca52907)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:06.058881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb6a1d-466b-4725-836f-c1c5534449a4 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:06.065957 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 114/216] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:05 2022] GET /identity/v3/users/b744e4892be341558b8f28271ca52907 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:06.066992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb6a1d-466b-4725-836f-c1c5534449a4 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:06.142784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73d391-4180-4978-8600-c276d20c7c50 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:06.180268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eeb43a-9a48-4a74-8124-13c9a3b8c84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:06.180897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeb43a-9a48-4a74-8124-13c9a3b8c84a None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.181205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeb43a-9a48-4a74-8124-13c9a3b8c84a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.181495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeb43a-9a48-4a74-8124-13c9a3b8c84a None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.182140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eeb43a-9a48-4a74-8124-13c9a3b8c84a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:06.183716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eeb43a-9a48-4a74-8124-13c9a3b8c84a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:06.258618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb6a1d-466b-4725-836f-c1c5534449a4 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:06.313645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 103/217] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:57:06 2022] POST /identity/v3/users => generated 297 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:06.319578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f6bd1-a824-480d-b686-c804c65c5e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:06.319813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752a0d1-1cb5-4a6c-97c5-ae331dbc43ae None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.320142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752a0d1-1cb5-4a6c-97c5-ae331dbc43ae None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.320370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752a0d1-1cb5-4a6c-97c5-ae331dbc43ae None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.321968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6bd1-a824-480d-b686-c804c65c5e42 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.322374 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 104/218] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:57:06.322728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6bd1-a824-480d-b686-c804c65c5e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.323145 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6bd1-a824-480d-b686-c804c65c5e42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.328822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6425b50-9b13-40a2-83ef-331014047788 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.329276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6425b50-9b13-40a2-83ef-331014047788 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.329652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6425b50-9b13-40a2-83ef-331014047788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.350079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-a6425b50-9b13-40a2-83ef-331014047788 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:06.351756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a6425b50-9b13-40a2-83ef-331014047788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:06.364900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:06.381321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:06.424159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f6bd1-a824-480d-b686-c804c65c5e42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:06.427359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f6bd1-a824-480d-b686-c804c65c5e42 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:06.454825 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 105/219] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:57:06.557671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 115/220] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:06.564890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6425b50-9b13-40a2-83ef-331014047788 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:06.708968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6425b50-9b13-40a2-83ef-331014047788 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:06.756830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b097-f27f-4cf5-b159-4832fc83514f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:06.757052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f6bd1-a824-480d-b686-c804c65c5e42 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:06.758157 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b097-f27f-4cf5-b159-4832fc83514f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.758449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b097-f27f-4cf5-b159-4832fc83514f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.758897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b097-f27f-4cf5-b159-4832fc83514f None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.761413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128b097-f27f-4cf5-b159-4832fc83514f None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:06.762889 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128b097-f27f-4cf5-b159-4832fc83514f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:06.764646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 106/221] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:06 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:06.771243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b097-f27f-4cf5-b159-4832fc83514f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:06.790584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:06.873290 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:06.912816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b097-f27f-4cf5-b159-4832fc83514f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:06.915760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f6bd1-a824-480d-b686-c804c65c5e42 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:06.954595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241e14b-c20d-4270-92e7-3a7daba3fdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:06.955451 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241e14b-c20d-4270-92e7-3a7daba3fdaa None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.955872 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241e14b-c20d-4270-92e7-3a7daba3fdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.956345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241e14b-c20d-4270-92e7-3a7daba3fdaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.957294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241e14b-c20d-4270-92e7-3a7daba3fdaa None admin] RBAC: Adding query filter params (role.id=13913230583d4ce384599ac5dace7ffa, scope.project.id=42ce709191ce4b1b961e89e218566b5b, user.id=b744e4892be341558b8f28271ca52907) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:06.957790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241e14b-c20d-4270-92e7-3a7daba3fdaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:06.959483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241e14b-c20d-4270-92e7-3a7daba3fdaa None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:06.962021 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8124c-daef-4ff1-98a3-457476fc50b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:06.962670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8124c-daef-4ff1-98a3-457476fc50b5 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.962953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8124c-daef-4ff1-98a3-457476fc50b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.963229 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8124c-daef-4ff1-98a3-457476fc50b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.969169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 107/222] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:06 2022] GET /identity/v3/role_assignments?user.id=b744e4892be341558b8f28271ca52907&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=13913230583d4ce384599ac5dace7ffa => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:06.970910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d1df0-a3ef-45f9-8ad1-d4dda0d343c7 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.971359 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d1df0-a3ef-45f9-8ad1-d4dda0d343c7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.971774 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d1df0-a3ef-45f9-8ad1-d4dda0d343c7 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.973824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 108/223] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:06.984951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd8e81-48c3-43ba-bcb1-d6de18e725ed None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.984951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd8e81-48c3-43ba-bcb1-d6de18e725ed None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.984951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd8e81-48c3-43ba-bcb1-d6de18e725ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.985340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8124c-daef-4ff1-98a3-457476fc50b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-49c8124c-daef-4ff1-98a3-457476fc50b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:06.989579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:06.991335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:06.992804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 116/224] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:57:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:06.998553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8124c-daef-4ff1-98a3-457476fc50b5 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:07.026816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b3cd8e81-48c3-43ba-bcb1-d6de18e725ed None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:07.028137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b3cd8e81-48c3-43ba-bcb1-d6de18e725ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:07.156268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 109/225] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:07.223102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8124c-daef-4ff1-98a3-457476fc50b5 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:07.273438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:07.281227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5aebb3-c595-41d8-9c31-d5665d70b5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:07.281875 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5aebb3-c595-41d8-9c31-d5665d70b5cd None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:07.282158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5aebb3-c595-41d8-9c31-d5665d70b5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:07.286380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5aebb3-c595-41d8-9c31-d5665d70b5cd None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:07.286380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5aebb3-c595-41d8-9c31-d5665d70b5cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:07.286380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5aebb3-c595-41d8-9c31-d5665d70b5cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:07.286380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5aebb3-c595-41d8-9c31-d5665d70b5cd None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:07.298337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 117/226] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:07.303653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd8e81-48c3-43ba-bcb1-d6de18e725ed None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:07.319307 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:07.531605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd8e81-48c3-43ba-bcb1-d6de18e725ed None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:07.583585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fb9d3-4868-4e0c-8848-3653f288ead1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:07.584314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb9d3-4868-4e0c-8848-3653f288ead1 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:07.584598 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb9d3-4868-4e0c-8848-3653f288ead1 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:07.584888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb9d3-4868-4e0c-8848-3653f288ead1 None admin] PATH_INFO: `/v3/users/47aad92357db4357b5b7b484dc866813` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:07.586299 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fb9d3-4868-4e0c-8848-3653f288ead1 None admin] RBAC: Authorizing `identity:get_user(user_id=47aad92357db4357b5b7b484dc866813)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:07.587759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fb9d3-4868-4e0c-8848-3653f288ead1 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:07.589474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 110/227] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:07 2022] GET /identity/v3/users/47aad92357db4357b5b7b484dc866813 => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:07.594996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5aebb3-c595-41d8-9c31-d5665d70b5cd None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:07.655080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fb9d3-4868-4e0c-8848-3653f288ead1 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:07.715350 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5aebb3-c595-41d8-9c31-d5665d70b5cd None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:07.763163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287859e6-8e8f-4b59-9c5f-f392cad59e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:07.763819 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287859e6-8e8f-4b59-9c5f-f392cad59e07 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:07.764075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287859e6-8e8f-4b59-9c5f-f392cad59e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:07.764353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287859e6-8e8f-4b59-9c5f-f392cad59e07 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:07.766012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287859e6-8e8f-4b59-9c5f-f392cad59e07 None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:07.767468 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287859e6-8e8f-4b59-9c5f-f392cad59e07 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:07.771174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 118/228] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:07 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:07.781799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287859e6-8e8f-4b59-9c5f-f392cad59e07 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:07.847060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:07.857202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:07.902242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287859e6-8e8f-4b59-9c5f-f392cad59e07 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:07.923749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fb9d3-4868-4e0c-8848-3653f288ead1 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:07.954713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd17f0-dd57-4693-933b-1772b6a0baf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:07.955386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd17f0-dd57-4693-933b-1772b6a0baf2 None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:07.955664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd17f0-dd57-4693-933b-1772b6a0baf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:07.955938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd17f0-dd57-4693-933b-1772b6a0baf2 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b/users/47aad92357db4357b5b7b484dc866813/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:07.959028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd17f0-dd57-4693-933b-1772b6a0baf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=42ce709191ce4b1b961e89e218566b5b, user_id=47aad92357db4357b5b7b484dc866813, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:07.960484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd17f0-dd57-4693-933b-1772b6a0baf2 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:07.978373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0715e7-bd81-46a1-a792-825265b5426b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:07.979015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0715e7-bd81-46a1-a792-825265b5426b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:07.979357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0715e7-bd81-46a1-a792-825265b5426b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:07.979760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0715e7-bd81-46a1-a792-825265b5426b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:07.990620 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0715e7-bd81-46a1-a792-825265b5426b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-5f0715e7-bd81-46a1-a792-825265b5426b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:07.996154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:07.997688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:07.998507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 111/229] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:57:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:57:08.005312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0715e7-bd81-46a1-a792-825265b5426b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:08.048190 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 119/230] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:57:07 2022] PUT /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b/users/47aad92357db4357b5b7b484dc866813/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:08.137263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0715e7-bd81-46a1-a792-825265b5426b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:08.194659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062c1fb-c5b3-4f3f-a237-2aec15f5a819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:08.197410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062c1fb-c5b3-4f3f-a237-2aec15f5a819 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:08.197903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062c1fb-c5b3-4f3f-a237-2aec15f5a819 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:08.198371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062c1fb-c5b3-4f3f-a237-2aec15f5a819 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:08.199378 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062c1fb-c5b3-4f3f-a237-2aec15f5a819 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:08.199880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062c1fb-c5b3-4f3f-a237-2aec15f5a819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:08.201646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062c1fb-c5b3-4f3f-a237-2aec15f5a819 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:08.210275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:08.219856 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 112/231] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:08.226016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd17f0-dd57-4693-933b-1772b6a0baf2 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:08.370658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd17f0-dd57-4693-933b-1772b6a0baf2 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:08.404275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:08.419821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29574d04-c535-4903-901d-414bdebf05fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:08.420510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574d04-c535-4903-901d-414bdebf05fc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:08.420802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574d04-c535-4903-901d-414bdebf05fc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:08.421147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574d04-c535-4903-901d-414bdebf05fc None admin] PATH_INFO: `/v3/users/47aad92357db4357b5b7b484dc866813` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:08.422458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29574d04-c535-4903-901d-414bdebf05fc None admin] RBAC: Authorizing `identity:get_user(user_id=47aad92357db4357b5b7b484dc866813)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:08.423971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29574d04-c535-4903-901d-414bdebf05fc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:08.426105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 120/232] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:08 2022] GET /identity/v3/users/47aad92357db4357b5b7b484dc866813 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:08.432527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062c1fb-c5b3-4f3f-a237-2aec15f5a819 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:08.582692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062c1fb-c5b3-4f3f-a237-2aec15f5a819 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:08.629868 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b75bb-1896-468a-a9d5-56b1169f0f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:08.630863 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b75bb-1896-468a-a9d5-56b1169f0f2a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:08.631351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b75bb-1896-468a-a9d5-56b1169f0f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:08.631832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b75bb-1896-468a-a9d5-56b1169f0f2a None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:08.633281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b75bb-1896-468a-a9d5-56b1169f0f2a None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:08.635172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b75bb-1896-468a-a9d5-56b1169f0f2a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:08.637443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 113/233] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:08 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:08.642809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29574d04-c535-4903-901d-414bdebf05fc None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:08.817446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25847160-9012-48f5-99c9-896b28bba4a0 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:08.818117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25847160-9012-48f5-99c9-896b28bba4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:08.818593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25847160-9012-48f5-99c9-896b28bba4a0 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:08.820193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 114/234] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:57:08.821465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29574d04-c535-4903-901d-414bdebf05fc None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:08.825562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285b589-dd11-4a79-ba0b-f8908b9e1d6d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:08.826074 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285b589-dd11-4a79-ba0b-f8908b9e1d6d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:08.826600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285b589-dd11-4a79-ba0b-f8908b9e1d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:08.857560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-1285b589-dd11-4a79-ba0b-f8908b9e1d6d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:08.860358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-1285b589-dd11-4a79-ba0b-f8908b9e1d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:08.896286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c741ed4-9dfc-4e2a-9242-3a9d82ff096f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:08.897359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c741ed4-9dfc-4e2a-9242-3a9d82ff096f None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:08.897909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c741ed4-9dfc-4e2a-9242-3a9d82ff096f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:08.898515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c741ed4-9dfc-4e2a-9242-3a9d82ff096f None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b/users/47aad92357db4357b5b7b484dc866813/roles/276ae07995d14b5d8ceb8e646392c2b1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:08.900966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c741ed4-9dfc-4e2a-9242-3a9d82ff096f None admin] RBAC: Authorizing `identity:create_grant(project_id=42ce709191ce4b1b961e89e218566b5b, user_id=47aad92357db4357b5b7b484dc866813, role_id=276ae07995d14b5d8ceb8e646392c2b1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:08.902858 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c741ed4-9dfc-4e2a-9242-3a9d82ff096f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:08.970930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:08.981318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 115/235] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:09.025816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 121/236] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:57:08 2022] PUT /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b/users/47aad92357db4357b5b7b484dc866813/roles/276ae07995d14b5d8ceb8e646392c2b1 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:09.088456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7923f-f6c4-459c-a72f-acab57c22d9c None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:09.089142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7923f-f6c4-459c-a72f-acab57c22d9c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:09.089663 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7923f-f6c4-459c-a72f-acab57c22d9c None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:09.091700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 116/237] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:09.097693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047fec2-c357-4603-89ca-7b315d707720 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:09.098253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047fec2-c357-4603-89ca-7b315d707720 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:09.098735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047fec2-c357-4603-89ca-7b315d707720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:09.120547 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-7047fec2-c357-4603-89ca-7b315d707720 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:09.122311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7047fec2-c357-4603-89ca-7b315d707720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:09.176948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:09.205030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 122/238] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:09.334280 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7923f-f6c4-459c-a72f-acab57c22d9c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:09.473019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7923f-f6c4-459c-a72f-acab57c22d9c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:09.502423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047fec2-c357-4603-89ca-7b315d707720 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:09.526840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f764cc8-4670-49f6-ab9e-80548bd1dcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:09.527905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f764cc8-4670-49f6-ab9e-80548bd1dcee None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:09.528463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f764cc8-4670-49f6-ab9e-80548bd1dcee None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:09.529032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f764cc8-4670-49f6-ab9e-80548bd1dcee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:09.542152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f764cc8-4670-49f6-ab9e-80548bd1dcee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-2f764cc8-4670-49f6-ab9e-80548bd1dcee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:09.547489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:09.548411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:09.549268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:09.552004 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 117/239] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:09.553578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef14185-8ad4-45e5-af4c-fe96811d31af None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:09.554102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef14185-8ad4-45e5-af4c-fe96811d31af None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:09.560421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef14185-8ad4-45e5-af4c-fe96811d31af None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:09.562171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 118/240] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:09.563111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef14185-8ad4-45e5-af4c-fe96811d31af None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:09.631357 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047fec2-c357-4603-89ca-7b315d707720 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:09.643701 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:09.672682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36be52a-5696-4f85-8200-46dacb6fc4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:09.673443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36be52a-5696-4f85-8200-46dacb6fc4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:09.673790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36be52a-5696-4f85-8200-46dacb6fc4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:09.674130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36be52a-5696-4f85-8200-46dacb6fc4b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:09.675360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36be52a-5696-4f85-8200-46dacb6fc4b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:09.676883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36be52a-5696-4f85-8200-46dacb6fc4b4 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:09.678718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 123/241] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:09.685322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aea80c-a676-4f9a-8c14-ea969773b82d None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:09.685690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aea80c-a676-4f9a-8c14-ea969773b82d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:09.686037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aea80c-a676-4f9a-8c14-ea969773b82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:09.701402 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef14185-8ad4-45e5-af4c-fe96811d31af None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:09.705344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-b6aea80c-a676-4f9a-8c14-ea969773b82d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:09.706595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b6aea80c-a676-4f9a-8c14-ea969773b82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:09.739633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2e3dc-2825-443a-9cec-a098e057a131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:09.740226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e3dc-2825-443a-9cec-a098e057a131 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:09.740519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e3dc-2825-443a-9cec-a098e057a131 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:09.740802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e3dc-2825-443a-9cec-a098e057a131 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:09.741629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2e3dc-2825-443a-9cec-a098e057a131 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:09.742287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2e3dc-2825-443a-9cec-a098e057a131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:09.743948 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2e3dc-2825-443a-9cec-a098e057a131 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:09.754181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 119/242] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:09.755092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:09.756559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2e3dc-2825-443a-9cec-a098e057a131 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:09.782414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 124/243] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:09.786990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aea80c-a676-4f9a-8c14-ea969773b82d None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:09.946651 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aea80c-a676-4f9a-8c14-ea969773b82d None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:09.970431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2e3dc-2825-443a-9cec-a098e057a131 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:10.006608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:10.028547 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8929932-9c60-42b5-97ac-7394aae08a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:10.029558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8929932-9c60-42b5-97ac-7394aae08a11 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:10.030023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8929932-9c60-42b5-97ac-7394aae08a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:10.030474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8929932-9c60-42b5-97ac-7394aae08a11 None admin] PATH_INFO: `/v3/users/5a463c32168f478abc75676d9f522cf5` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:10.043264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71d9f3-1a58-4a87-9389-15e09ae1a4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:10.044541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71d9f3-1a58-4a87-9389-15e09ae1a4e9 None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:10.045721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71d9f3-1a58-4a87-9389-15e09ae1a4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:10.045987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71d9f3-1a58-4a87-9389-15e09ae1a4e9 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:10.046628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71d9f3-1a58-4a87-9389-15e09ae1a4e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:10.048496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71d9f3-1a58-4a87-9389-15e09ae1a4e9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:10.058683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8929932-9c60-42b5-97ac-7394aae08a11 None admin] RBAC: Authorizing `identity:get_user(user_id=5a463c32168f478abc75676d9f522cf5)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:10.060517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8929932-9c60-42b5-97ac-7394aae08a11 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:10.062741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 125/244] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:09 2022] GET /identity/v3/users/5a463c32168f478abc75676d9f522cf5 => generated 297 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:10.071230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8929932-9c60-42b5-97ac-7394aae08a11 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:10.114084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:10.150079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 120/245] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:57:09 2022] POST /identity/v3/users => generated 293 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:10.150079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71d9f3-1a58-4a87-9389-15e09ae1a4e9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:10.241560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8929932-9c60-42b5-97ac-7394aae08a11 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:10.340438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8e45e-4a26-4888-bde1-902a6fa54375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:10.341253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8e45e-4a26-4888-bde1-902a6fa54375 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:10.341543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8e45e-4a26-4888-bde1-902a6fa54375 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:10.341831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8e45e-4a26-4888-bde1-902a6fa54375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:10.360434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71d9f3-1a58-4a87-9389-15e09ae1a4e9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:10.409419 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27047ed3-cdbf-4091-bad8-591160eedcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:10.410462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27047ed3-cdbf-4091-bad8-591160eedcfe None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:10.411252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27047ed3-cdbf-4091-bad8-591160eedcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:10.411534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27047ed3-cdbf-4091-bad8-591160eedcfe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:10.428983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27047ed3-cdbf-4091-bad8-591160eedcfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-27047ed3-cdbf-4091-bad8-591160eedcfe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:10.437102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:10.438021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:10.438805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:10.439995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 121/246] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:10.440073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8e45e-4a26-4888-bde1-902a6fa54375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:10.441633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8e45e-4a26-4888-bde1-902a6fa54375 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:10.446639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27047ed3-cdbf-4091-bad8-591160eedcfe None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:10.489226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:10.494218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:10.575425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27047ed3-cdbf-4091-bad8-591160eedcfe None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:10.586037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 126/247] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:10.587582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfe821-0c93-4c29-ae7b-e1b03d5b0c9e None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:10.588024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfe821-0c93-4c29-ae7b-e1b03d5b0c9e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:10.588298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfe821-0c93-4c29-ae7b-e1b03d5b0c9e None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:10.589743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 127/248] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:10.595972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2990bf5-5794-4fad-9184-d7ebe8b93c36 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:10.596469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2990bf5-5794-4fad-9184-d7ebe8b93c36 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:10.597112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2990bf5-5794-4fad-9184-d7ebe8b93c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:10.624583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-f2990bf5-5794-4fad-9184-d7ebe8b93c36 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:10.625806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f2990bf5-5794-4fad-9184-d7ebe8b93c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:10.629763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3587a1-ba67-47d0-acf7-f754c4e67d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:10.631415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3587a1-ba67-47d0-acf7-f754c4e67d06 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:10.631415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3587a1-ba67-47d0-acf7-f754c4e67d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:10.631415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3587a1-ba67-47d0-acf7-f754c4e67d06 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:10.631678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3587a1-ba67-47d0-acf7-f754c4e67d06 None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:10.631903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3587a1-ba67-47d0-acf7-f754c4e67d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:10.633920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3587a1-ba67-47d0-acf7-f754c4e67d06 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:10.644596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 122/249] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:10.645274 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3587a1-ba67-47d0-acf7-f754c4e67d06 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:10.715727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 128/250] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:57:10.717603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2990bf5-5794-4fad-9184-d7ebe8b93c36 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:10.761623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:10.767674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3587a1-ba67-47d0-acf7-f754c4e67d06 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:10.805248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:10.822994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbea55-5b32-41d1-8205-c006e8e43985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:10.824654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbea55-5b32-41d1-8205-c006e8e43985 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:10.824654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbea55-5b32-41d1-8205-c006e8e43985 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:10.824654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbea55-5b32-41d1-8205-c006e8e43985 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:10.826396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbea55-5b32-41d1-8205-c006e8e43985 None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:10.827984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbea55-5b32-41d1-8205-c006e8e43985 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:10.830213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 123/251] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:10 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:10.837527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbea55-5b32-41d1-8205-c006e8e43985 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:10.883040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2990bf5-5794-4fad-9184-d7ebe8b93c36 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:10.958245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbea55-5b32-41d1-8205-c006e8e43985 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:10.969584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3236f2-ef52-4185-9b4f-7badade6f8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:10.970247 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3236f2-ef52-4185-9b4f-7badade6f8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:10.970514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3236f2-ef52-4185-9b4f-7badade6f8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:10.970781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3236f2-ef52-4185-9b4f-7badade6f8d7 None admin] PATH_INFO: `/v3/users/b744e4892be341558b8f28271ca52907` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:10.973002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3236f2-ef52-4185-9b4f-7badade6f8d7 None admin] RBAC: Authorizing `identity:get_user(user_id=b744e4892be341558b8f28271ca52907)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:10.974461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3236f2-ef52-4185-9b4f-7badade6f8d7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:10.977089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 129/252] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:10 2022] GET /identity/v3/users/b744e4892be341558b8f28271ca52907 => generated 297 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:10.984068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3236f2-ef52-4185-9b4f-7badade6f8d7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:10.998267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5942480-e3ec-4eb5-859a-a0a1e6ea2315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:10.998885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5942480-e3ec-4eb5-859a-a0a1e6ea2315 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:10.999167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5942480-e3ec-4eb5-859a-a0a1e6ea2315 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:10.999439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5942480-e3ec-4eb5-859a-a0a1e6ea2315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:11.000696 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5942480-e3ec-4eb5-859a-a0a1e6ea2315 None admin] RBAC: Adding query filter params (role.id=b9b25895899d46eea0c912f6277165cb, scope.project.id=42ce709191ce4b1b961e89e218566b5b, user.id=5a463c32168f478abc75676d9f522cf5) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:11.001045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5942480-e3ec-4eb5-859a-a0a1e6ea2315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:11.002584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5942480-e3ec-4eb5-859a-a0a1e6ea2315 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:11.010188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 124/253] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:10 2022] GET /identity/v3/role_assignments?user.id=5a463c32168f478abc75676d9f522cf5&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=b9b25895899d46eea0c912f6277165cb => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:11.159786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3236f2-ef52-4185-9b4f-7badade6f8d7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:11.206498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83577892-5e7f-4c16-b837-fdacc6155501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:11.206760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83577892-5e7f-4c16-b837-fdacc6155501 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:11.206912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83577892-5e7f-4c16-b837-fdacc6155501 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:11.207232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83577892-5e7f-4c16-b837-fdacc6155501 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:11.213852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83577892-5e7f-4c16-b837-fdacc6155501 None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:11.213852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83577892-5e7f-4c16-b837-fdacc6155501 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:11.214422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 130/254] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:10 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:11.219780 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5942480-e3ec-4eb5-859a-a0a1e6ea2315 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:11.251999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:11.262001 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83577892-5e7f-4c16-b837-fdacc6155501 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:11.284355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:11.359911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5942480-e3ec-4eb5-859a-a0a1e6ea2315 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:11.453667 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6560c1-40ae-4761-b1fa-2abfc295a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:11.454284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6560c1-40ae-4761-b1fa-2abfc295a12d None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:11.454541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6560c1-40ae-4761-b1fa-2abfc295a12d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:11.454823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6560c1-40ae-4761-b1fa-2abfc295a12d None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b/users/b744e4892be341558b8f28271ca52907/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:11.461636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6560c1-40ae-4761-b1fa-2abfc295a12d None admin] RBAC: Authorizing `identity:create_grant(project_id=42ce709191ce4b1b961e89e218566b5b, user_id=b744e4892be341558b8f28271ca52907, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:11.463103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6560c1-40ae-4761-b1fa-2abfc295a12d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:11.510988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83577892-5e7f-4c16-b837-fdacc6155501 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:11.555001 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 125/255] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:57:11 2022] PUT /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b/users/b744e4892be341558b8f28271ca52907/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:11.575148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445efb95-3375-414a-9bcc-a2d4feb89563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:11.575811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445efb95-3375-414a-9bcc-a2d4feb89563 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:11.576094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445efb95-3375-414a-9bcc-a2d4feb89563 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:11.576372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445efb95-3375-414a-9bcc-a2d4feb89563 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:11.596336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445efb95-3375-414a-9bcc-a2d4feb89563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-445efb95-3375-414a-9bcc-a2d4feb89563 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:11.602208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:11.603184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:11.603959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 131/256] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:57:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:11.609397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6560c1-40ae-4761-b1fa-2abfc295a12d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:11.747521 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:11.773101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6560c1-40ae-4761-b1fa-2abfc295a12d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:11.831669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf11a80-bc72-4909-a4c8-2d30abbdec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:11.832699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf11a80-bc72-4909-a4c8-2d30abbdec7b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:11.833227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf11a80-bc72-4909-a4c8-2d30abbdec7b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:11.833704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf11a80-bc72-4909-a4c8-2d30abbdec7b None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:11.834586 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf11a80-bc72-4909-a4c8-2d30abbdec7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:11.835112 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf11a80-bc72-4909-a4c8-2d30abbdec7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:11.837064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf11a80-bc72-4909-a4c8-2d30abbdec7b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:11.847755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 126/257] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:11.854111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445efb95-3375-414a-9bcc-a2d4feb89563 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:11.914832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f8c9f-7554-49ab-bfcc-79da9084b667 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:11.915527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f8c9f-7554-49ab-bfcc-79da9084b667 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:11.916029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f8c9f-7554-49ab-bfcc-79da9084b667 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:11.917925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 127/258] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:11.922785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa80b7-8da9-432d-ba8a-446b79bfeead None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:11.923308 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa80b7-8da9-432d-ba8a-446b79bfeead None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:11.923822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa80b7-8da9-432d-ba8a-446b79bfeead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:11.951660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-6aaa80b7-8da9-432d-ba8a-446b79bfeead None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:11.954434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6aaa80b7-8da9-432d-ba8a-446b79bfeead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:11.967817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:12.056433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445efb95-3375-414a-9bcc-a2d4feb89563 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:12.131044 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 128/259] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:57:12.131418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082fbf8-e914-4e03-97d6-d46641133953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:12.132003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082fbf8-e914-4e03-97d6-d46641133953 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:12.132897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082fbf8-e914-4e03-97d6-d46641133953 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:12.133185 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082fbf8-e914-4e03-97d6-d46641133953 None admin] PATH_INFO: `/v3/users/47aad92357db4357b5b7b484dc866813` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:12.134346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082fbf8-e914-4e03-97d6-d46641133953 None admin] RBAC: Authorizing `identity:get_user(user_id=47aad92357db4357b5b7b484dc866813)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:12.135831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082fbf8-e914-4e03-97d6-d46641133953 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:12.143015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa80b7-8da9-432d-ba8a-446b79bfeead None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:12.145278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 132/260] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:11 2022] GET /identity/v3/users/47aad92357db4357b5b7b484dc866813 => generated 283 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:12.281103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:12.305191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa80b7-8da9-432d-ba8a-446b79bfeead None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:12.410313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef02ee-3b1b-440e-883f-0ea788168674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:12.410929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef02ee-3b1b-440e-883f-0ea788168674 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:12.411201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef02ee-3b1b-440e-883f-0ea788168674 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:12.411467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef02ee-3b1b-440e-883f-0ea788168674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:12.503693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef02ee-3b1b-440e-883f-0ea788168674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:12.505189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef02ee-3b1b-440e-883f-0ea788168674 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:12.621466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:12.645984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 129/261] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:57:12.652415 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082fbf8-e914-4e03-97d6-d46641133953 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:12.687350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef02ee-3b1b-440e-883f-0ea788168674 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:12.815714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082fbf8-e914-4e03-97d6-d46641133953 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:12.863722 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:12.869802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3757f5-996f-43a6-93cc-99cd8f8bd4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:12.872879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3757f5-996f-43a6-93cc-99cd8f8bd4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:12.872879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3757f5-996f-43a6-93cc-99cd8f8bd4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:12.872879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3757f5-996f-43a6-93cc-99cd8f8bd4e7 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:12.872879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3757f5-996f-43a6-93cc-99cd8f8bd4e7 None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:12.873578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3757f5-996f-43a6-93cc-99cd8f8bd4e7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:12.875368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 133/262] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:12 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:12.881141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3757f5-996f-43a6-93cc-99cd8f8bd4e7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:12.895707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef02ee-3b1b-440e-883f-0ea788168674 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:12.960038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e92c9d-29c4-4107-ac91-65a9ffab21b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:12.963216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e92c9d-29c4-4107-ac91-65a9ffab21b0 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:12.963833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e92c9d-29c4-4107-ac91-65a9ffab21b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:12.964307 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e92c9d-29c4-4107-ac91-65a9ffab21b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:12.987432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e92c9d-29c4-4107-ac91-65a9ffab21b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-d8e92c9d-29c4-4107-ac91-65a9ffab21b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:12.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:12.994269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:12.995079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:12.995692 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3757f5-996f-43a6-93cc-99cd8f8bd4e7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:13.002996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 130/263] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:57:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:57:13.009985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e92c9d-29c4-4107-ac91-65a9ffab21b0 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:13.049049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17664df1-036d-40be-8ac0-9394facc2924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:13.050015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17664df1-036d-40be-8ac0-9394facc2924 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:13.050476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17664df1-036d-40be-8ac0-9394facc2924 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:13.050939 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17664df1-036d-40be-8ac0-9394facc2924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:13.052356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17664df1-036d-40be-8ac0-9394facc2924 None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=42ce709191ce4b1b961e89e218566b5b, user.id=47aad92357db4357b5b7b484dc866813) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:13.052982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17664df1-036d-40be-8ac0-9394facc2924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:13.058941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17664df1-036d-40be-8ac0-9394facc2924 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:13.064408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 134/264] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:12 2022] GET /identity/v3/role_assignments?user.id=47aad92357db4357b5b7b484dc866813&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:13.103823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:13.212204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e92c9d-29c4-4107-ac91-65a9ffab21b0 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:13.278450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8a4d2-c7c7-406e-9ffd-3fc798c8ae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:13.279317 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8a4d2-c7c7-406e-9ffd-3fc798c8ae5b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:13.279714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8a4d2-c7c7-406e-9ffd-3fc798c8ae5b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:13.280091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8a4d2-c7c7-406e-9ffd-3fc798c8ae5b None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:13.281169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8a4d2-c7c7-406e-9ffd-3fc798c8ae5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:13.281624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8a4d2-c7c7-406e-9ffd-3fc798c8ae5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:13.283728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8a4d2-c7c7-406e-9ffd-3fc798c8ae5b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:13.299692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 131/265] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:13.306926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17664df1-036d-40be-8ac0-9394facc2924 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:13.330079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16471495-2bdf-44fd-b9be-f77a326265e3 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:13.330423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16471495-2bdf-44fd-b9be-f77a326265e3 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:13.330636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16471495-2bdf-44fd-b9be-f77a326265e3 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:13.333110 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 132/266] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:13.341394 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c459fdb-9a66-4d19-8936-958b40511bb3 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:13.341909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c459fdb-9a66-4d19-8936-958b40511bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:13.342418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c459fdb-9a66-4d19-8936-958b40511bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:13.385757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-5c459fdb-9a66-4d19-8936-958b40511bb3 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:13.387258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5c459fdb-9a66-4d19-8936-958b40511bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:13.433265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:13.488045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17664df1-036d-40be-8ac0-9394facc2924 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:13.529532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 133/267] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:13.545477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e0c73-82e4-4199-ae61-8025bf210cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:13.546141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e0c73-82e4-4199-ae61-8025bf210cdf None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:13.546433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e0c73-82e4-4199-ae61-8025bf210cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:13.546718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e0c73-82e4-4199-ae61-8025bf210cdf None admin] PATH_INFO: `/v3/users/47aad92357db4357b5b7b484dc866813` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:13.547882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e0c73-82e4-4199-ae61-8025bf210cdf None admin] RBAC: Authorizing `identity:get_user(user_id=47aad92357db4357b5b7b484dc866813)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:13.549444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e0c73-82e4-4199-ae61-8025bf210cdf None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:13.551135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 135/268] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:13 2022] GET /identity/v3/users/47aad92357db4357b5b7b484dc866813 => generated 283 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:13.557962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c459fdb-9a66-4d19-8936-958b40511bb3 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:13.568385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5cac7-4333-4edf-9cb5-b4672a384d82 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:13.568726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5cac7-4333-4edf-9cb5-b4672a384d82 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:13.568987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5cac7-4333-4edf-9cb5-b4672a384d82 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:13.571197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 136/269] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:13.577178 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f858b-b5b5-4101-b108-40f23297812c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:13.577696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f858b-b5b5-4101-b108-40f23297812c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:13.578157 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f858b-b5b5-4101-b108-40f23297812c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:13.606145 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-2a9f858b-b5b5-4101-b108-40f23297812c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:13.607910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2a9f858b-b5b5-4101-b108-40f23297812c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:13.638379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:13.690547 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 137/270] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:57:13.745930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c459fdb-9a66-4d19-8936-958b40511bb3 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:13.798593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc73c33-186b-4ea0-bdf1-32831cb78b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:13.799225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc73c33-186b-4ea0-bdf1-32831cb78b21 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:13.799503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc73c33-186b-4ea0-bdf1-32831cb78b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:13.799777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc73c33-186b-4ea0-bdf1-32831cb78b21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:13.866094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a2318-0017-4581-9e2e-74c1b3971af7 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:13.867034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a2318-0017-4581-9e2e-74c1b3971af7 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:13.867506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a2318-0017-4581-9e2e-74c1b3971af7 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:13.869267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 138/271] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:57:13.874416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68eccd-b1cc-4dff-9d48-2161b8744466 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:13.874695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68eccd-b1cc-4dff-9d48-2161b8744466 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:13.874968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68eccd-b1cc-4dff-9d48-2161b8744466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:13.900228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc73c33-186b-4ea0-bdf1-32831cb78b21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:13.901559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-cd68eccd-b1cc-4dff-9d48-2161b8744466 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:13.901753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc73c33-186b-4ea0-bdf1-32831cb78b21 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:13.902758 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-cd68eccd-b1cc-4dff-9d48-2161b8744466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:13.956196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:13.973555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:13.979189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 139/272] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:57:14.036675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68eccd-b1cc-4dff-9d48-2161b8744466 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:14.086938 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 134/273] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:14.087824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc73c33-186b-4ea0-bdf1-32831cb78b21 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:14.178017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68eccd-b1cc-4dff-9d48-2161b8744466 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:14.216067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabac55a-6f47-4640-8681-43c257d66153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:14.216705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabac55a-6f47-4640-8681-43c257d66153 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.217015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabac55a-6f47-4640-8681-43c257d66153 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.217291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabac55a-6f47-4640-8681-43c257d66153 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.218393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabac55a-6f47-4640-8681-43c257d66153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:14.219835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabac55a-6f47-4640-8681-43c257d66153 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:14.222192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc73c33-186b-4ea0-bdf1-32831cb78b21 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:14.226816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 140/274] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:14.227723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabac55a-6f47-4640-8681-43c257d66153 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:14.264846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:14.277219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadb831-63c8-4863-bcc2-ba724bea6d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:14.277868 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadb831-63c8-4863-bcc2-ba724bea6d0f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.278146 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadb831-63c8-4863-bcc2-ba724bea6d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.278420 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadb831-63c8-4863-bcc2-ba724bea6d0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.297090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadb831-63c8-4863-bcc2-ba724bea6d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:14.297901 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-1fadb831-63c8-4863-bcc2-ba724bea6d0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:14.302255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:14.303366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:14.304753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 135/275] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:14.306622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadb831-63c8-4863-bcc2-ba724bea6d0f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:14.343105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabac55a-6f47-4640-8681-43c257d66153 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:14.380673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb80f73-3e53-4d90-b988-10cbd1d56f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:14.381373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb80f73-3e53-4d90-b988-10cbd1d56f82 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.381771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb80f73-3e53-4d90-b988-10cbd1d56f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.382005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb80f73-3e53-4d90-b988-10cbd1d56f82 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.384217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb80f73-3e53-4d90-b988-10cbd1d56f82 None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:14.385702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb80f73-3e53-4d90-b988-10cbd1d56f82 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:14.387356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 141/276] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:14 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:14.391744 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb80f73-3e53-4d90-b988-10cbd1d56f82 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:14.429882 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadb831-63c8-4863-bcc2-ba724bea6d0f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:14.472155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be424008-5507-4693-a85a-b7737553c87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:14.473134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be424008-5507-4693-a85a-b7737553c87c None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.473598 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be424008-5507-4693-a85a-b7737553c87c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.474050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be424008-5507-4693-a85a-b7737553c87c None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.474896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be424008-5507-4693-a85a-b7737553c87c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:14.478268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be424008-5507-4693-a85a-b7737553c87c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:14.506782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb80f73-3e53-4d90-b988-10cbd1d56f82 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:14.533051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:14.542446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d9525-0e70-4fee-a3ca-ded62446e3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:14.543067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d9525-0e70-4fee-a3ca-ded62446e3ff None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.543341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d9525-0e70-4fee-a3ca-ded62446e3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.543608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d9525-0e70-4fee-a3ca-ded62446e3ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.553653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d9525-0e70-4fee-a3ca-ded62446e3ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-bb9d9525-0e70-4fee-a3ca-ded62446e3ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:14.558721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:14.559836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 142/277] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:14.560985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d9525-0e70-4fee-a3ca-ded62446e3ff None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:14.657492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d9525-0e70-4fee-a3ca-ded62446e3ff None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:14.663064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 136/278] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:57:14 2022] POST /identity/v3/users => generated 293 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:14.663064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be424008-5507-4693-a85a-b7737553c87c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:14.701632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab993559-2101-44ba-9dcf-cd5c163fb4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:14.702253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab993559-2101-44ba-9dcf-cd5c163fb4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.704399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab993559-2101-44ba-9dcf-cd5c163fb4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.704513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab993559-2101-44ba-9dcf-cd5c163fb4f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.705159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab993559-2101-44ba-9dcf-cd5c163fb4f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:14.705549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab993559-2101-44ba-9dcf-cd5c163fb4f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:14.707135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab993559-2101-44ba-9dcf-cd5c163fb4f1 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:14.718241 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 143/279] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:14.719455 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabd459-3d2e-4376-a49b-14d6f6a02317 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.719743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabd459-3d2e-4376-a49b-14d6f6a02317 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.720018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabd459-3d2e-4376-a49b-14d6f6a02317 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.721453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 144/280] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:14.727405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabd459-3d2e-4376-a49b-14d6f6a02317 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:14.746012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:14.786713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:14.814937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be424008-5507-4693-a85a-b7737553c87c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:14.864804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e9d16-7c88-4f97-b348-09c94842c031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:14.865779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e9d16-7c88-4f97-b348-09c94842c031 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.866070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e9d16-7c88-4f97-b348-09c94842c031 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.866336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabd459-3d2e-4376-a49b-14d6f6a02317 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:14.866502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e9d16-7c88-4f97-b348-09c94842c031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.867658 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e9d16-7c88-4f97-b348-09c94842c031 None admin] RBAC: Adding query filter params (role.id=276ae07995d14b5d8ceb8e646392c2b1, scope.project.id=42ce709191ce4b1b961e89e218566b5b, user.id=47aad92357db4357b5b7b484dc866813) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:14.868111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e9d16-7c88-4f97-b348-09c94842c031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:14.870557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e9d16-7c88-4f97-b348-09c94842c031 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:14.879998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 137/281] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:14 2022] GET /identity/v3/role_assignments?user.id=47aad92357db4357b5b7b484dc866813&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=276ae07995d14b5d8ceb8e646392c2b1 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:14.881384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e9d16-7c88-4f97-b348-09c94842c031 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:14.912712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ab8da-63e1-4665-9639-3c0d2b951f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:14.913419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ab8da-63e1-4665-9639-3c0d2b951f3a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.913715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ab8da-63e1-4665-9639-3c0d2b951f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.915505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ab8da-63e1-4665-9639-3c0d2b951f3a None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.916234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ab8da-63e1-4665-9639-3c0d2b951f3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:14.916709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ab8da-63e1-4665-9639-3c0d2b951f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:14.918197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ab8da-63e1-4665-9639-3c0d2b951f3a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:14.928778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 145/282] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:14.930154 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f285ee6-b812-419c-86c2-c93046f9e49c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.930442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f285ee6-b812-419c-86c2-c93046f9e49c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.930722 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f285ee6-b812-419c-86c2-c93046f9e49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.953992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-4f285ee6-b812-419c-86c2-c93046f9e49c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:14.955878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-4f285ee6-b812-419c-86c2-c93046f9e49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:15.038045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e9d16-7c88-4f97-b348-09c94842c031 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:15.051437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 146/283] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:15.052331 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f285ee6-b812-419c-86c2-c93046f9e49c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:15.057650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:15.089974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdba69-d904-4472-98d6-3c4265dd772a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:15.090604 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdba69-d904-4472-98d6-3c4265dd772a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:15.091099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdba69-d904-4472-98d6-3c4265dd772a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:15.091435 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdba69-d904-4472-98d6-3c4265dd772a None admin] PATH_INFO: `/v3/users/9e46027cd2bf431883f87b6e9bec2f0a` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:15.112899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdba69-d904-4472-98d6-3c4265dd772a None admin] RBAC: Authorizing `identity:get_user(user_id=9e46027cd2bf431883f87b6e9bec2f0a)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:15.115499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:15.115953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdba69-d904-4472-98d6-3c4265dd772a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:15.123793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 138/284] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:14 2022] GET /identity/v3/users/9e46027cd2bf431883f87b6e9bec2f0a => generated 293 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:15.125806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdba69-d904-4472-98d6-3c4265dd772a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:15.200180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f285ee6-b812-419c-86c2-c93046f9e49c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:15.269358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41df3ac-c975-44a7-8745-675d812f5aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:15.270032 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41df3ac-c975-44a7-8745-675d812f5aba None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:15.270309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41df3ac-c975-44a7-8745-675d812f5aba None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:15.270586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41df3ac-c975-44a7-8745-675d812f5aba None admin] PATH_INFO: `/v3/users/5a463c32168f478abc75676d9f522cf5` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:15.274322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41df3ac-c975-44a7-8745-675d812f5aba None admin] RBAC: Authorizing `identity:get_user(user_id=5a463c32168f478abc75676d9f522cf5)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:15.276366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41df3ac-c975-44a7-8745-675d812f5aba None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:15.278120 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 147/285] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:15 2022] GET /identity/v3/users/5a463c32168f478abc75676d9f522cf5 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:15.292969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41df3ac-c975-44a7-8745-675d812f5aba None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:15.317851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdba69-d904-4472-98d6-3c4265dd772a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:15.373047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599089f5-da0f-4a55-bf3b-a02571bfb4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:15.374459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599089f5-da0f-4a55-bf3b-a02571bfb4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:15.374949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599089f5-da0f-4a55-bf3b-a02571bfb4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:15.375443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599089f5-da0f-4a55-bf3b-a02571bfb4d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:15.376384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599089f5-da0f-4a55-bf3b-a02571bfb4d2 None admin] RBAC: Adding query filter params (role.id=13913230583d4ce384599ac5dace7ffa, user.id=9e46027cd2bf431883f87b6e9bec2f0a, scope.system=all) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:15.377515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599089f5-da0f-4a55-bf3b-a02571bfb4d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:15.379658 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599089f5-da0f-4a55-bf3b-a02571bfb4d2 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:15.395466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 139/286] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Aug 24 09:57:15 2022] GET /identity/v3/role_assignments?user.id=9e46027cd2bf431883f87b6e9bec2f0a&scope.system=all&role.id=13913230583d4ce384599ac5dace7ffa => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:15.468035 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41df3ac-c975-44a7-8745-675d812f5aba None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:15.485204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:15.487523 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599089f5-da0f-4a55-bf3b-a02571bfb4d2 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:15.514357 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135776b-efc8-40b3-8beb-d438cfe26bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:15.515012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135776b-efc8-40b3-8beb-d438cfe26bfe None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:15.515286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135776b-efc8-40b3-8beb-d438cfe26bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:15.515554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135776b-efc8-40b3-8beb-d438cfe26bfe None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:15.516785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135776b-efc8-40b3-8beb-d438cfe26bfe None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:15.518294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135776b-efc8-40b3-8beb-d438cfe26bfe None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:15.520013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 148/287] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:15 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:15.527116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135776b-efc8-40b3-8beb-d438cfe26bfe None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:15.580527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:15.641551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135776b-efc8-40b3-8beb-d438cfe26bfe None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:15.680892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2703f24-1fbd-4051-8476-889575cc08ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:15.681753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2703f24-1fbd-4051-8476-889575cc08ab None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:15.682226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2703f24-1fbd-4051-8476-889575cc08ab None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:15.682683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2703f24-1fbd-4051-8476-889575cc08ab None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b/users/5a463c32168f478abc75676d9f522cf5/roles/b9b25895899d46eea0c912f6277165cb` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:15.684888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2703f24-1fbd-4051-8476-889575cc08ab None admin] RBAC: Authorizing `identity:create_grant(project_id=42ce709191ce4b1b961e89e218566b5b, user_id=5a463c32168f478abc75676d9f522cf5, role_id=b9b25895899d46eea0c912f6277165cb)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:15.686480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2703f24-1fbd-4051-8476-889575cc08ab None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:15.729273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599089f5-da0f-4a55-bf3b-a02571bfb4d2 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:15.773967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 149/288] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:57:15 2022] PUT /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b/users/5a463c32168f478abc75676d9f522cf5/roles/b9b25895899d46eea0c912f6277165cb => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:15.774694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ec449-6e21-42cd-8fd5-87bb7a00fb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:15.775388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ec449-6e21-42cd-8fd5-87bb7a00fb7c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:15.775709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ec449-6e21-42cd-8fd5-87bb7a00fb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:15.775985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ec449-6e21-42cd-8fd5-87bb7a00fb7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:15.796907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ec449-6e21-42cd-8fd5-87bb7a00fb7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-bc9ec449-6e21-42cd-8fd5-87bb7a00fb7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:15.802205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:15.803105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:15.804071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 140/289] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:15.811478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2703f24-1fbd-4051-8476-889575cc08ab None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:15.868484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:15.956292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2703f24-1fbd-4051-8476-889575cc08ab None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:16.009951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa65a0-ee06-4ab2-8966-c17b20d68ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:16.010586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa65a0-ee06-4ab2-8966-c17b20d68ade None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:16.010872 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa65a0-ee06-4ab2-8966-c17b20d68ade None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:16.011148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa65a0-ee06-4ab2-8966-c17b20d68ade None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:16.011837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa65a0-ee06-4ab2-8966-c17b20d68ade None admin] RBAC: Adding query filter params (name=member) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:16.012144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa65a0-ee06-4ab2-8966-c17b20d68ade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:16.013676 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa65a0-ee06-4ab2-8966-c17b20d68ade None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:16.029470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 150/290] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:16.035383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ec449-6e21-42cd-8fd5-87bb7a00fb7c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:16.157743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:16.166178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ec449-6e21-42cd-8fd5-87bb7a00fb7c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:16.213571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c588e-c1a8-431c-95ba-b94c7d2136c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:16.214125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c588e-c1a8-431c-95ba-b94c7d2136c5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:16.214405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c588e-c1a8-431c-95ba-b94c7d2136c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:16.214678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c588e-c1a8-431c-95ba-b94c7d2136c5 None admin] PATH_INFO: `/v3/users/b744e4892be341558b8f28271ca52907` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:16.215886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c588e-c1a8-431c-95ba-b94c7d2136c5 None admin] RBAC: Authorizing `identity:get_user(user_id=b744e4892be341558b8f28271ca52907)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:16.217377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c588e-c1a8-431c-95ba-b94c7d2136c5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:16.219378 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 141/291] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:16 2022] GET /identity/v3/users/b744e4892be341558b8f28271ca52907 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:16.224801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa65a0-ee06-4ab2-8966-c17b20d68ade None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:16.354165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:16.358143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa65a0-ee06-4ab2-8966-c17b20d68ade None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:16.394258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eadb78-fbc3-48e9-9c25-84d89fc85053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:16.394870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eadb78-fbc3-48e9-9c25-84d89fc85053 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:16.395151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eadb78-fbc3-48e9-9c25-84d89fc85053 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:16.395450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eadb78-fbc3-48e9-9c25-84d89fc85053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:16.467140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eadb78-fbc3-48e9-9c25-84d89fc85053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:16.468550 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eadb78-fbc3-48e9-9c25-84d89fc85053 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:16.569948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:16.574748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 151/292] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:16.580426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c588e-c1a8-431c-95ba-b94c7d2136c5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:16.719865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c588e-c1a8-431c-95ba-b94c7d2136c5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:16.771115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738083f-c47c-45e4-9521-0db103f0b4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:16.771886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738083f-c47c-45e4-9521-0db103f0b4ac None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:16.772125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738083f-c47c-45e4-9521-0db103f0b4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:16.772344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738083f-c47c-45e4-9521-0db103f0b4ac None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:16.774398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738083f-c47c-45e4-9521-0db103f0b4ac None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:16.775880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738083f-c47c-45e4-9521-0db103f0b4ac None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:16.777590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 142/293] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:16 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:16.783692 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eadb78-fbc3-48e9-9c25-84d89fc85053 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:16.907226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eadb78-fbc3-48e9-9c25-84d89fc85053 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:16.945198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92baf61-dd01-46ce-a2ef-2251f8d4d444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:16.945823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92baf61-dd01-46ce-a2ef-2251f8d4d444 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:16.946268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92baf61-dd01-46ce-a2ef-2251f8d4d444 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:16.946549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92baf61-dd01-46ce-a2ef-2251f8d4d444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:16.947284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92baf61-dd01-46ce-a2ef-2251f8d4d444 None admin] RBAC: Adding query filter params (role.id=13913230583d4ce384599ac5dace7ffa, scope.project.id=42ce709191ce4b1b961e89e218566b5b, user.id=b744e4892be341558b8f28271ca52907) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:16.947617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92baf61-dd01-46ce-a2ef-2251f8d4d444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:16.949088 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92baf61-dd01-46ce-a2ef-2251f8d4d444 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:16.955673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 152/294] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:16 2022] GET /identity/v3/role_assignments?user.id=b744e4892be341558b8f28271ca52907&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=13913230583d4ce384599ac5dace7ffa => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:17.351626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d580c3f-ecce-4e68-a904-7097b05fe684 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:17.352627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d580c3f-ecce-4e68-a904-7097b05fe684 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:17.352975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d580c3f-ecce-4e68-a904-7097b05fe684 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:17.355019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 143/295] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:17.360954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01db1bf-e8a0-4711-baf3-437355b37db7 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:17.361452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01db1bf-e8a0-4711-baf3-437355b37db7 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:17.361529 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01db1bf-e8a0-4711-baf3-437355b37db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:17.380908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a01db1bf-e8a0-4711-baf3-437355b37db7 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:17.382866 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a01db1bf-e8a0-4711-baf3-437355b37db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:17.458329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:17.472988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 153/296] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:17.681881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed091aa-94db-401c-b5ad-3831adb39f04 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:17.682264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed091aa-94db-401c-b5ad-3831adb39f04 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:17.682437 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed091aa-94db-401c-b5ad-3831adb39f04 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:17.683820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 144/297] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:17.689576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81932-d00c-497d-846d-5b7011d502d1 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:17.689906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81932-d00c-497d-846d-5b7011d502d1 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:17.690138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81932-d00c-497d-846d-5b7011d502d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:17.710942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-d2e81932-d00c-497d-846d-5b7011d502d1 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:17.712165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d2e81932-d00c-497d-846d-5b7011d502d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:17.775594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed091aa-94db-401c-b5ad-3831adb39f04 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:17.791179 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 154/298] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:17.811934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:17.858515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a36d48-1cb5-404e-9369-e8fabdabfd17 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:17.858810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a36d48-1cb5-404e-9369-e8fabdabfd17 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:17.859265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a36d48-1cb5-404e-9369-e8fabdabfd17 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:17.860850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 155/299] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:57:17.867516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b46678-8527-4459-a871-eb0a008bbe32 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:17.868217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b46678-8527-4459-a871-eb0a008bbe32 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:17.868558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b46678-8527-4459-a871-eb0a008bbe32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:17.887902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-d4b46678-8527-4459-a871-eb0a008bbe32 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:17.890352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d4b46678-8527-4459-a871-eb0a008bbe32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:17.902362 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed091aa-94db-401c-b5ad-3831adb39f04 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:17.954527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78712e3e-491c-42ff-9ae8-46af6d7e2cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:17.955080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712e3e-491c-42ff-9ae8-46af6d7e2cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:17.955360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712e3e-491c-42ff-9ae8-46af6d7e2cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:17.955699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712e3e-491c-42ff-9ae8-46af6d7e2cf3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:17.972841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78712e3e-491c-42ff-9ae8-46af6d7e2cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-78712e3e-491c-42ff-9ae8-46af6d7e2cf3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:17.980926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:17.982054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:17.983079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:17.983899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 156/301] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:57:17.984899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 145/301] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:57:17.986408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b46678-8527-4459-a871-eb0a008bbe32 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:18.065502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:18.082878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78712e3e-491c-42ff-9ae8-46af6d7e2cf3 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:18.115618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b46678-8527-4459-a871-eb0a008bbe32 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:18.164658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ba67c-1df4-432d-86b4-f25e8bd1a810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:18.165319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ba67c-1df4-432d-86b4-f25e8bd1a810 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:18.165603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ba67c-1df4-432d-86b4-f25e8bd1a810 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:18.165933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ba67c-1df4-432d-86b4-f25e8bd1a810 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:18.166711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ba67c-1df4-432d-86b4-f25e8bd1a810 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:18.167020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ba67c-1df4-432d-86b4-f25e8bd1a810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:18.168760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ba67c-1df4-432d-86b4-f25e8bd1a810 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:18.183258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 157/302] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:18.185959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ba67c-1df4-432d-86b4-f25e8bd1a810 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:18.211993 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:18.226577 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78712e3e-491c-42ff-9ae8-46af6d7e2cf3 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:18.275014 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50772e-fcfd-4105-a661-9e0cf37c71bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:18.275936 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50772e-fcfd-4105-a661-9e0cf37c71bd None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:18.276382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50772e-fcfd-4105-a661-9e0cf37c71bd None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:18.276800 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50772e-fcfd-4105-a661-9e0cf37c71bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:18.278186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50772e-fcfd-4105-a661-9e0cf37c71bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:18.279966 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50772e-fcfd-4105-a661-9e0cf37c71bd None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:18.281786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 146/303] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:18.282593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50772e-fcfd-4105-a661-9e0cf37c71bd None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:18.311712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ba67c-1df4-432d-86b4-f25e8bd1a810 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:18.328016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:18.348164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf8722-884f-4604-99cf-5dac13d4ee4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:18.348788 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf8722-884f-4604-99cf-5dac13d4ee4d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:18.349093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf8722-884f-4604-99cf-5dac13d4ee4d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:18.349368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf8722-884f-4604-99cf-5dac13d4ee4d None admin] PATH_INFO: `/v3/users/4ff30a51660841b4bca2ef3de6ede040` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:18.388556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf8722-884f-4604-99cf-5dac13d4ee4d None admin] RBAC: Authorizing `identity:get_user(user_id=4ff30a51660841b4bca2ef3de6ede040)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:18.390064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf8722-884f-4604-99cf-5dac13d4ee4d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:18.391757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 158/304] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:18 2022] GET /identity/v3/users/4ff30a51660841b4bca2ef3de6ede040 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:18.397221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf8722-884f-4604-99cf-5dac13d4ee4d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:18.397332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50772e-fcfd-4105-a661-9e0cf37c71bd None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:18.431688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fbab2-69c4-4003-aef3-771913b8ab21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:18.432342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fbab2-69c4-4003-aef3-771913b8ab21 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:18.432623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fbab2-69c4-4003-aef3-771913b8ab21 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:18.432939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fbab2-69c4-4003-aef3-771913b8ab21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:18.443116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fbab2-69c4-4003-aef3-771913b8ab21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-c80fbab2-69c4-4003-aef3-771913b8ab21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:18.448133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:18.449098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:18.449910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 147/305] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:18.450567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3a294-01ef-4db4-9182-6bd81ff3dfad None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:18.450981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3a294-01ef-4db4-9182-6bd81ff3dfad None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:18.451678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3a294-01ef-4db4-9182-6bd81ff3dfad None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:18.452450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 148/306] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:18.453211 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3a294-01ef-4db4-9182-6bd81ff3dfad None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:18.474327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:18.501708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf8722-884f-4604-99cf-5dac13d4ee4d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:18.537791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6808e9-f2fa-4f76-b07e-9bb14c0e44fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:18.539062 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6808e9-f2fa-4f76-b07e-9bb14c0e44fb None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:18.539591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6808e9-f2fa-4f76-b07e-9bb14c0e44fb None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:18.539852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6808e9-f2fa-4f76-b07e-9bb14c0e44fb None admin] PATH_INFO: `/v3/groups` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:18.540472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6808e9-f2fa-4f76-b07e-9bb14c0e44fb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:18.541924 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6808e9-f2fa-4f76-b07e-9bb14c0e44fb None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:18.566389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3a294-01ef-4db4-9182-6bd81ff3dfad None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:18.605776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225eb6f-2ff8-4e0b-87a1-1d51b5ca7777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:18.606396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225eb6f-2ff8-4e0b-87a1-1d51b5ca7777 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:18.606674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225eb6f-2ff8-4e0b-87a1-1d51b5ca7777 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:18.606944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225eb6f-2ff8-4e0b-87a1-1d51b5ca7777 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:18.607713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225eb6f-2ff8-4e0b-87a1-1d51b5ca7777 None admin] RBAC: Adding query filter params (role.id=b9b25895899d46eea0c912f6277165cb, user.id=4ff30a51660841b4bca2ef3de6ede040, scope.system=all) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:18.608020 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225eb6f-2ff8-4e0b-87a1-1d51b5ca7777 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:18.609542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225eb6f-2ff8-4e0b-87a1-1d51b5ca7777 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:18.614702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 149/307] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Aug 24 09:57:18 2022] GET /identity/v3/role_assignments?user.id=4ff30a51660841b4bca2ef3de6ede040&scope.system=all&role.id=b9b25895899d46eea0c912f6277165cb => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:18.615497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225eb6f-2ff8-4e0b-87a1-1d51b5ca7777 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:18.630213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 159/308] 10.222.0.30 () {60 vars in 1134 bytes} [Wed Aug 24 09:57:18 2022] POST /identity/v3/groups => generated 229 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:18.631590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607f8ba-f29a-4b94-9ec2-b9126c9f4b30 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:18.632017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607f8ba-f29a-4b94-9ec2-b9126c9f4b30 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:18.632437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607f8ba-f29a-4b94-9ec2-b9126c9f4b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:18.654939 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c607f8ba-f29a-4b94-9ec2-b9126c9f4b30 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:18.656297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c607f8ba-f29a-4b94-9ec2-b9126c9f4b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:18.701251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:18.725189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:18.747472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225eb6f-2ff8-4e0b-87a1-1d51b5ca7777 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:18.749423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 160/309] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:18.799425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183a101-acab-41ab-a6e3-1c9436cbdf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:18.800087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183a101-acab-41ab-a6e3-1c9436cbdf96 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:18.800396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183a101-acab-41ab-a6e3-1c9436cbdf96 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:18.800670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183a101-acab-41ab-a6e3-1c9436cbdf96 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:18.801404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183a101-acab-41ab-a6e3-1c9436cbdf96 None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:18.801709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183a101-acab-41ab-a6e3-1c9436cbdf96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:18.803169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183a101-acab-41ab-a6e3-1c9436cbdf96 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:18.813264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 150/310] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:18.818514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607f8ba-f29a-4b94-9ec2-b9126c9f4b30 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:18.939857 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607f8ba-f29a-4b94-9ec2-b9126c9f4b30 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:18.981121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2819e-3bf8-4607-98a0-72cd1083d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:18.981683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2819e-3bf8-4607-98a0-72cd1083d82d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:18.981966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2819e-3bf8-4607-98a0-72cd1083d82d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:18.982293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2819e-3bf8-4607-98a0-72cd1083d82d None admin] PATH_INFO: `/v3/users/9e46027cd2bf431883f87b6e9bec2f0a` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:18.983465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2819e-3bf8-4607-98a0-72cd1083d82d None admin] RBAC: Authorizing `identity:get_user(user_id=9e46027cd2bf431883f87b6e9bec2f0a)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:18.984949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2819e-3bf8-4607-98a0-72cd1083d82d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:18.987094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 161/311] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:18 2022] GET /identity/v3/users/9e46027cd2bf431883f87b6e9bec2f0a => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:18.994149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183a101-acab-41ab-a6e3-1c9436cbdf96 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:19.028254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2819e-3bf8-4607-98a0-72cd1083d82d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:19.069067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:19.111823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183a101-acab-41ab-a6e3-1c9436cbdf96 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:19.149702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9977d6-09de-4471-a7af-97dda25a78fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:19.150356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9977d6-09de-4471-a7af-97dda25a78fd None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:19.150634 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9977d6-09de-4471-a7af-97dda25a78fd None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:19.150905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9977d6-09de-4471-a7af-97dda25a78fd None admin] PATH_INFO: `/v3/system/users/9e46027cd2bf431883f87b6e9bec2f0a/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:19.152534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9977d6-09de-4471-a7af-97dda25a78fd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9e46027cd2bf431883f87b6e9bec2f0a, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:19.154047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9977d6-09de-4471-a7af-97dda25a78fd None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:19.165733 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2819e-3bf8-4607-98a0-72cd1083d82d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:19.204968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30736590-95f6-4403-ba4d-8d5bbb702b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:19.205607 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30736590-95f6-4403-ba4d-8d5bbb702b7c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:19.205887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30736590-95f6-4403-ba4d-8d5bbb702b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:19.206163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30736590-95f6-4403-ba4d-8d5bbb702b7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:19.217088 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30736590-95f6-4403-ba4d-8d5bbb702b7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-30736590-95f6-4403-ba4d-8d5bbb702b7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:19.222108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:19.222993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:19.223770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 162/312] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:57:19.230269 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30736590-95f6-4403-ba4d-8d5bbb702b7c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:19.269153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 151/313] 10.222.0.30 () {58 vars in 1334 bytes} [Wed Aug 24 09:57:18 2022] PUT /identity/v3/system/users/9e46027cd2bf431883f87b6e9bec2f0a/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:19.352893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:19.365223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30736590-95f6-4403-ba4d-8d5bbb702b7c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:19.405505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc7cc0-d804-4329-96e4-aef6f9d37df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:19.406300 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc7cc0-d804-4329-96e4-aef6f9d37df9 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:19.406522 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc7cc0-d804-4329-96e4-aef6f9d37df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:19.406796 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc7cc0-d804-4329-96e4-aef6f9d37df9 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:19.407527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc7cc0-d804-4329-96e4-aef6f9d37df9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:19.407830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc7cc0-d804-4329-96e4-aef6f9d37df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:19.409250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc7cc0-d804-4329-96e4-aef6f9d37df9 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:19.420101 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 163/314] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:19.425969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9977d6-09de-4471-a7af-97dda25a78fd None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:19.437956 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:19.536968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9977d6-09de-4471-a7af-97dda25a78fd None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:19.574035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98bd16-e5c5-458f-a4c1-091fdf65d954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:19.574932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98bd16-e5c5-458f-a4c1-091fdf65d954 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:19.575358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98bd16-e5c5-458f-a4c1-091fdf65d954 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:19.575773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98bd16-e5c5-458f-a4c1-091fdf65d954 None admin] PATH_INFO: `/v3/users/5a463c32168f478abc75676d9f522cf5` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:19.577075 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98bd16-e5c5-458f-a4c1-091fdf65d954 None admin] RBAC: Authorizing `identity:get_user(user_id=5a463c32168f478abc75676d9f522cf5)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:19.578660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98bd16-e5c5-458f-a4c1-091fdf65d954 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:19.580379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 152/315] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:19 2022] GET /identity/v3/users/5a463c32168f478abc75676d9f522cf5 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:19.593144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc7cc0-d804-4329-96e4-aef6f9d37df9 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:19.704041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc7cc0-d804-4329-96e4-aef6f9d37df9 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:19.740362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb559f7-c095-448a-bce6-9e00d1670d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:19.741394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb559f7-c095-448a-bce6-9e00d1670d91 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:19.741850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb559f7-c095-448a-bce6-9e00d1670d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:19.742299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb559f7-c095-448a-bce6-9e00d1670d91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:19.767778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:19.816716 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb559f7-c095-448a-bce6-9e00d1670d91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:19.818493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb559f7-c095-448a-bce6-9e00d1670d91 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:19.932097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 164/316] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:19.933339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98bd16-e5c5-458f-a4c1-091fdf65d954 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:20.017684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:20.045325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98bd16-e5c5-458f-a4c1-091fdf65d954 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:20.080416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38337cd2-f8c0-41dd-af01-f8a683e0aca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:20.081020 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38337cd2-f8c0-41dd-af01-f8a683e0aca8 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:20.081305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38337cd2-f8c0-41dd-af01-f8a683e0aca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:20.081585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38337cd2-f8c0-41dd-af01-f8a683e0aca8 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:20.082878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38337cd2-f8c0-41dd-af01-f8a683e0aca8 None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:20.084316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38337cd2-f8c0-41dd-af01-f8a683e0aca8 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:20.086888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 153/317] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:19 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:20.096967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb559f7-c095-448a-bce6-9e00d1670d91 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:20.167438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:20.214398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb559f7-c095-448a-bce6-9e00d1670d91 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:20.254928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2e82a-9cb1-410c-ac1d-8d476fe60d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:20.255988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e82a-9cb1-410c-ac1d-8d476fe60d2d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:20.256463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e82a-9cb1-410c-ac1d-8d476fe60d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:20.257184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e82a-9cb1-410c-ac1d-8d476fe60d2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:20.258162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2e82a-9cb1-410c-ac1d-8d476fe60d2d None admin] RBAC: Adding query filter params (role.id=b9b25895899d46eea0c912f6277165cb, scope.project.id=42ce709191ce4b1b961e89e218566b5b, user.id=5a463c32168f478abc75676d9f522cf5) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:20.258667 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2e82a-9cb1-410c-ac1d-8d476fe60d2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:20.260339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2e82a-9cb1-410c-ac1d-8d476fe60d2d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:20.267066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 165/318] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:20 2022] GET /identity/v3/role_assignments?user.id=5a463c32168f478abc75676d9f522cf5&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=b9b25895899d46eea0c912f6277165cb => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:20.664628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57386ef-7b36-4c8d-9f2b-6a9ac5fd5b65 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:20.664965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57386ef-7b36-4c8d-9f2b-6a9ac5fd5b65 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:20.665276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57386ef-7b36-4c8d-9f2b-6a9ac5fd5b65 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:20.666825 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 154/319] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:20.672235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cadc4a-02d3-4b38-a6b2-cf0bad4cc352 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:20.672730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cadc4a-02d3-4b38-a6b2-cf0bad4cc352 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:20.672813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cadc4a-02d3-4b38-a6b2-cf0bad4cc352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:20.691468 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-b7cadc4a-02d3-4b38-a6b2-cf0bad4cc352 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:20.692662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b7cadc4a-02d3-4b38-a6b2-cf0bad4cc352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:20.762553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe1e97-a299-43bb-90c7-0ae623cfe296 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:20.762830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe1e97-a299-43bb-90c7-0ae623cfe296 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:20.763103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe1e97-a299-43bb-90c7-0ae623cfe296 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:20.764434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 155/320] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:20.769759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 166/321] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:20.771161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e1829-188f-4579-a5bd-0211f8996f3c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:20.771445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e1829-188f-4579-a5bd-0211f8996f3c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:20.771715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e1829-188f-4579-a5bd-0211f8996f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:20.791093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-c95e1829-188f-4579-a5bd-0211f8996f3c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:20.792309 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c95e1829-188f-4579-a5bd-0211f8996f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:20.864903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 156/322] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:21.051516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cadc4a-02d3-4b38-a6b2-cf0bad4cc352 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:21.100112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:21.151487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e1829-188f-4579-a5bd-0211f8996f3c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:21.164952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cadc4a-02d3-4b38-a6b2-cf0bad4cc352 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:21.204829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34175a4f-e3a4-4a71-bd55-a2d6f9908d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:21.205521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34175a4f-e3a4-4a71-bd55-a2d6f9908d67 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:21.205803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34175a4f-e3a4-4a71-bd55-a2d6f9908d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:21.206078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34175a4f-e3a4-4a71-bd55-a2d6f9908d67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:21.218622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34175a4f-e3a4-4a71-bd55-a2d6f9908d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-34175a4f-e3a4-4a71-bd55-a2d6f9908d67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:21.223841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:21.224768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:21.226003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 167/323] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:21.231382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34175a4f-e3a4-4a71-bd55-a2d6f9908d67 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:21.232783 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:21.260011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e1829-188f-4579-a5bd-0211f8996f3c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:21.306758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7002e29-020f-408f-8247-49e0487fa03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:21.307321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7002e29-020f-408f-8247-49e0487fa03e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:21.307599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7002e29-020f-408f-8247-49e0487fa03e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:21.307869 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7002e29-020f-408f-8247-49e0487fa03e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:21.308992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7002e29-020f-408f-8247-49e0487fa03e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:21.310426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7002e29-020f-408f-8247-49e0487fa03e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:21.312098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 157/324] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:21.313212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b155d-976f-474c-94e1-2e96afd4b236 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:21.318945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b155d-976f-474c-94e1-2e96afd4b236 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:21.318945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b155d-976f-474c-94e1-2e96afd4b236 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:21.318945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 158/325] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:21.318945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b155d-976f-474c-94e1-2e96afd4b236 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:21.356601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34175a4f-e3a4-4a71-bd55-a2d6f9908d67 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:21.364094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:21.391888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141257e8-59b2-4de5-ad9d-3b10642df060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:21.392610 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141257e8-59b2-4de5-ad9d-3b10642df060 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:21.392915 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141257e8-59b2-4de5-ad9d-3b10642df060 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:21.393199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141257e8-59b2-4de5-ad9d-3b10642df060 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:21.394008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141257e8-59b2-4de5-ad9d-3b10642df060 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:21.394313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141257e8-59b2-4de5-ad9d-3b10642df060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:21.395704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141257e8-59b2-4de5-ad9d-3b10642df060 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:21.404551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 168/326] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:21.410437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532a02-96d1-4441-9e70-5dab8f024aa1 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:21.410751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532a02-96d1-4441-9e70-5dab8f024aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:21.410990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532a02-96d1-4441-9e70-5dab8f024aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:21.425524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b155d-976f-474c-94e1-2e96afd4b236 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:21.428843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-27532a02-96d1-4441-9e70-5dab8f024aa1 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:21.430035 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-27532a02-96d1-4441-9e70-5dab8f024aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:21.459684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a891b01-88d8-4252-a8e2-a1e434dd89b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:21.461158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a891b01-88d8-4252-a8e2-a1e434dd89b8 None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:21.461452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a891b01-88d8-4252-a8e2-a1e434dd89b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:21.461727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a891b01-88d8-4252-a8e2-a1e434dd89b8 None admin] PATH_INFO: `/v3/groups` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:21.462351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a891b01-88d8-4252-a8e2-a1e434dd89b8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:21.463776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a891b01-88d8-4252-a8e2-a1e434dd89b8 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:21.496436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 169/327] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:21.497304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27532a02-96d1-4441-9e70-5dab8f024aa1 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:21.577683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 159/328] 10.222.0.30 () {60 vars in 1134 bytes} [Wed Aug 24 09:57:21 2022] POST /identity/v3/groups => generated 226 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:57:21.601749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:21.606904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27532a02-96d1-4441-9e70-5dab8f024aa1 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:21.646798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b16593-e23b-4dcd-af47-57f0d9b092dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:21.647470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b16593-e23b-4dcd-af47-57f0d9b092dc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:21.647755 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b16593-e23b-4dcd-af47-57f0d9b092dc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:21.648093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b16593-e23b-4dcd-af47-57f0d9b092dc None admin] PATH_INFO: `/v3/users/4ff30a51660841b4bca2ef3de6ede040` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:21.649475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b16593-e23b-4dcd-af47-57f0d9b092dc None admin] RBAC: Authorizing `identity:get_user(user_id=4ff30a51660841b4bca2ef3de6ede040)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:21.651155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b16593-e23b-4dcd-af47-57f0d9b092dc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:21.653100 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 170/329] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:21 2022] GET /identity/v3/users/4ff30a51660841b4bca2ef3de6ede040 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:21.659016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a891b01-88d8-4252-a8e2-a1e434dd89b8 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:21.674092 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:21.771975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b16593-e23b-4dcd-af47-57f0d9b092dc None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:21.790394 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a891b01-88d8-4252-a8e2-a1e434dd89b8 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:21.836995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceda08e-5c2d-408d-b96c-0ee2c811b97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:21.838477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda08e-5c2d-408d-b96c-0ee2c811b97e None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:21.838767 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda08e-5c2d-408d-b96c-0ee2c811b97e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:21.839046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda08e-5c2d-408d-b96c-0ee2c811b97e None admin] PATH_INFO: `/v3/system/users/4ff30a51660841b4bca2ef3de6ede040/roles/b9b25895899d46eea0c912f6277165cb` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:21.840500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceda08e-5c2d-408d-b96c-0ee2c811b97e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4ff30a51660841b4bca2ef3de6ede040, role_id=b9b25895899d46eea0c912f6277165cb)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:21.842035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceda08e-5c2d-408d-b96c-0ee2c811b97e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:21.892306 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b16593-e23b-4dcd-af47-57f0d9b092dc None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:21.932258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2bedf-2092-4418-89e6-8121a09b9c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:21.933431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2bedf-2092-4418-89e6-8121a09b9c6a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:21.933721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2bedf-2092-4418-89e6-8121a09b9c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:21.933994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2bedf-2092-4418-89e6-8121a09b9c6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:21.944517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 160/330] 10.222.0.30 () {58 vars in 1334 bytes} [Wed Aug 24 09:57:21 2022] PUT /identity/v3/system/users/4ff30a51660841b4bca2ef3de6ede040/roles/b9b25895899d46eea0c912f6277165cb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2bedf-2092-4418-89e6-8121a09b9c6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-5ca2bedf-2092-4418-89e6-8121a09b9c6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:21.954664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:21.955589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:21.956604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 171/331] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:21.959068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceda08e-5c2d-408d-b96c-0ee2c811b97e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:22.052414 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:22.096609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceda08e-5c2d-408d-b96c-0ee2c811b97e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:22.142920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621bbff-3374-4881-aacc-821148aeab60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:22.143944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621bbff-3374-4881-aacc-821148aeab60 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:22.144490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621bbff-3374-4881-aacc-821148aeab60 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:22.144998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621bbff-3374-4881-aacc-821148aeab60 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:22.145899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621bbff-3374-4881-aacc-821148aeab60 None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:22.146427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621bbff-3374-4881-aacc-821148aeab60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:22.148117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621bbff-3374-4881-aacc-821148aeab60 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:22.161864 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 161/332] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:22.168172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2bedf-2092-4418-89e6-8121a09b9c6a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:22.205266 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:22.294467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2bedf-2092-4418-89e6-8121a09b9c6a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:22.334484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5de64-ae57-4084-bc42-25faf6eb68db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:22.335663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5de64-ae57-4084-bc42-25faf6eb68db None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:22.335939 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5de64-ae57-4084-bc42-25faf6eb68db None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:22.336205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5de64-ae57-4084-bc42-25faf6eb68db None admin] PATH_INFO: `/v3/users/9e46027cd2bf431883f87b6e9bec2f0a` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:22.337383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5de64-ae57-4084-bc42-25faf6eb68db None admin] RBAC: Authorizing `identity:get_user(user_id=9e46027cd2bf431883f87b6e9bec2f0a)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:22.338863 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5de64-ae57-4084-bc42-25faf6eb68db None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:22.340757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 172/333] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:22 2022] GET /identity/v3/users/9e46027cd2bf431883f87b6e9bec2f0a => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:22.346861 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621bbff-3374-4881-aacc-821148aeab60 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:22.545119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621bbff-3374-4881-aacc-821148aeab60 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:22.578489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:22.589495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd35169-4a4e-48e9-8b72-07aeed08fe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:22.590165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd35169-4a4e-48e9-8b72-07aeed08fe6f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:22.594906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd35169-4a4e-48e9-8b72-07aeed08fe6f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:22.594906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd35169-4a4e-48e9-8b72-07aeed08fe6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:22.594906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd35169-4a4e-48e9-8b72-07aeed08fe6f None admin] RBAC: Adding query filter params (role.id=13913230583d4ce384599ac5dace7ffa, user.id=9e46027cd2bf431883f87b6e9bec2f0a, scope.system=all) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:22.594906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd35169-4a4e-48e9-8b72-07aeed08fe6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:22.594906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd35169-4a4e-48e9-8b72-07aeed08fe6f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:22.607459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 162/334] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Aug 24 09:57:22 2022] GET /identity/v3/role_assignments?user.id=9e46027cd2bf431883f87b6e9bec2f0a&scope.system=all&role.id=13913230583d4ce384599ac5dace7ffa => generated 510 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:24.211361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78e4f6-ba8e-4902-bafb-6a55a1b6f071 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:24.211914 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78e4f6-ba8e-4902-bafb-6a55a1b6f071 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:24.212188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78e4f6-ba8e-4902-bafb-6a55a1b6f071 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:24.213766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 173/335] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:24.222262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050609f9-d75d-4c33-ab37-e9fc78b4af88 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:24.222703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050609f9-d75d-4c33-ab37-e9fc78b4af88 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:24.222795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050609f9-d75d-4c33-ab37-e9fc78b4af88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:24.246553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-050609f9-d75d-4c33-ab37-e9fc78b4af88 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:24.247762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-050609f9-d75d-4c33-ab37-e9fc78b4af88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:24.331694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 163/336] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:24.699806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db94fa-434d-40a1-baaa-7e514f63eb9e None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:24.700486 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db94fa-434d-40a1-baaa-7e514f63eb9e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:24.700993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db94fa-434d-40a1-baaa-7e514f63eb9e None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:24.702881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 174/337] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:24.707976 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e502d3-aa2c-4b39-9c32-fcf4398b6514 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:24.708531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e502d3-aa2c-4b39-9c32-fcf4398b6514 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:24.709043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e502d3-aa2c-4b39-9c32-fcf4398b6514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:24.733288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-03e502d3-aa2c-4b39-9c32-fcf4398b6514 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:24.734892 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-03e502d3-aa2c-4b39-9c32-fcf4398b6514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:24.779985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c668e7-3001-4f5d-bdc3-77f8995d070d None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:24.780844 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c668e7-3001-4f5d-bdc3-77f8995d070d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:24.781627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c668e7-3001-4f5d-bdc3-77f8995d070d None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:24.784789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 175/338] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:24.796553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679df0e1-422f-4bf1-a66b-365f4a08a11a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:24.797450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679df0e1-422f-4bf1-a66b-365f4a08a11a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:24.798137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679df0e1-422f-4bf1-a66b-365f4a08a11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:24.863760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-679df0e1-422f-4bf1-a66b-365f4a08a11a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:24.866294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-679df0e1-422f-4bf1-a66b-365f4a08a11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:24.939369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 164/339] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:24.994322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e502d3-aa2c-4b39-9c32-fcf4398b6514 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:25.009067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 176/340] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:25.011025 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bfcb2-160c-456c-b12a-4f0e1d42ed51 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:25.011025 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bfcb2-160c-456c-b12a-4f0e1d42ed51 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:25.011252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bfcb2-160c-456c-b12a-4f0e1d42ed51 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:25.012729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 177/341] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:25.018193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ecab4-7d95-49f3-847c-c57452a52a50 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:25.018483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ecab4-7d95-49f3-847c-c57452a52a50 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:25.018822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ecab4-7d95-49f3-847c-c57452a52a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:25.051092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-ee7ecab4-7d95-49f3-847c-c57452a52a50 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:25.052220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ee7ecab4-7d95-49f3-847c-c57452a52a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:25.056991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:25.083947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:25.127059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e502d3-aa2c-4b39-9c32-fcf4398b6514 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:25.173172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 178/342] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:57:25.176590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a55f68-2ebe-41fa-819f-6d3fab280b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:25.177258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a55f68-2ebe-41fa-819f-6d3fab280b7c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:25.177535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a55f68-2ebe-41fa-819f-6d3fab280b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:25.177807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a55f68-2ebe-41fa-819f-6d3fab280b7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:25.197755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a55f68-2ebe-41fa-819f-6d3fab280b7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-45a55f68-2ebe-41fa-819f-6d3fab280b7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:25.204386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:25.205364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:25.206147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:25.206747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 165/343] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:25.213366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ecab4-7d95-49f3-847c-c57452a52a50 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:25.358013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ecab4-7d95-49f3-847c-c57452a52a50 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:25.400751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9a29d-8dfc-4a9e-bc1b-ca240627476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:25.402025 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9a29d-8dfc-4a9e-bc1b-ca240627476f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:25.402721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9a29d-8dfc-4a9e-bc1b-ca240627476f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:25.403416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9a29d-8dfc-4a9e-bc1b-ca240627476f None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:25.404475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9a29d-8dfc-4a9e-bc1b-ca240627476f None admin] RBAC: Adding query filter params (name=member) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:25.406199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a55f68-2ebe-41fa-819f-6d3fab280b7c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:25.407922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9a29d-8dfc-4a9e-bc1b-ca240627476f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:25.409894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9a29d-8dfc-4a9e-bc1b-ca240627476f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:25.420344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 179/344] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:25.426226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9a29d-8dfc-4a9e-bc1b-ca240627476f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:25.461913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:25.617404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9a29d-8dfc-4a9e-bc1b-ca240627476f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:25.620853 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a55f68-2ebe-41fa-819f-6d3fab280b7c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:25.632636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:25.690961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0f7e6-caa0-4016-b94b-bbc0534566c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:25.691588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0f7e6-caa0-4016-b94b-bbc0534566c5 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:25.691872 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0f7e6-caa0-4016-b94b-bbc0534566c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:25.692149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0f7e6-caa0-4016-b94b-bbc0534566c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:25.699937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01b286-6eb1-492e-b121-b17bd3de0b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:25.700367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01b286-6eb1-492e-b121-b17bd3de0b83 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:25.700645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01b286-6eb1-492e-b121-b17bd3de0b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:25.700937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01b286-6eb1-492e-b121-b17bd3de0b83 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:25.715940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01b286-6eb1-492e-b121-b17bd3de0b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-fa01b286-6eb1-492e-b121-b17bd3de0b83 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:25.720792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:25.721687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:25.722474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 166/345] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:57:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:25.723194 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01b286-6eb1-492e-b121-b17bd3de0b83 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:25.805731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0f7e6-caa0-4016-b94b-bbc0534566c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:25.807317 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0f7e6-caa0-4016-b94b-bbc0534566c5 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:25.829230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:25.873696 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01b286-6eb1-492e-b121-b17bd3de0b83 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:25.917188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843290c-b551-4892-b8b9-690fb62db014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:25.918160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843290c-b551-4892-b8b9-690fb62db014 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:25.918668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843290c-b551-4892-b8b9-690fb62db014 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:25.919124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843290c-b551-4892-b8b9-690fb62db014 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:25.933207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843290c-b551-4892-b8b9-690fb62db014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-6843290c-b551-4892-b8b9-690fb62db014 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:25.938939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:25.939988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:57:25.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:25.943275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 167/346] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:25.944729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843290c-b551-4892-b8b9-690fb62db014 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:25.945446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 180/347] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:57:25.946242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0f7e6-caa0-4016-b94b-bbc0534566c5 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:25.979974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:26.080479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843290c-b551-4892-b8b9-690fb62db014 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:26.080827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0f7e6-caa0-4016-b94b-bbc0534566c5 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:26.105801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:26.125363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d96c87-e8fa-461b-b4a4-c5ece32aab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:26.125955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d96c87-e8fa-461b-b4a4-c5ece32aab5a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:26.126231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d96c87-e8fa-461b-b4a4-c5ece32aab5a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:26.126506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d96c87-e8fa-461b-b4a4-c5ece32aab5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:26.134408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c31fed-eca7-4171-92c3-8e23bdbefd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:26.135273 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c31fed-eca7-4171-92c3-8e23bdbefd32 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:26.135502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c31fed-eca7-4171-92c3-8e23bdbefd32 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:26.135775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c31fed-eca7-4171-92c3-8e23bdbefd32 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:26.136629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c31fed-eca7-4171-92c3-8e23bdbefd32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:26.137395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c31fed-eca7-4171-92c3-8e23bdbefd32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:26.138831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c31fed-eca7-4171-92c3-8e23bdbefd32 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:26.140463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d96c87-e8fa-461b-b4a4-c5ece32aab5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-02d96c87-e8fa-461b-b4a4-c5ece32aab5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:26.145817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:26.146744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:26.148239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:26.150769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 168/348] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:26.152902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2150d-5aaa-4cb9-be01-35d20d04824a None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:26.153391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2150d-5aaa-4cb9-be01-35d20d04824a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:26.153835 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2150d-5aaa-4cb9-be01-35d20d04824a None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:26.154109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 181/349] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:26.154965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c31fed-eca7-4171-92c3-8e23bdbefd32 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:26.155911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 169/350] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:26.161074 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2150d-5aaa-4cb9-be01-35d20d04824a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:26.266000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:26.287749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2150d-5aaa-4cb9-be01-35d20d04824a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:26.299985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c31fed-eca7-4171-92c3-8e23bdbefd32 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:26.345190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4975d-8961-4d30-8826-13156c87f286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:26.345433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722838f-9677-4549-8540-33cd56964329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:26.345844 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722838f-9677-4549-8540-33cd56964329 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:26.345957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4975d-8961-4d30-8826-13156c87f286 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:26.346132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722838f-9677-4549-8540-33cd56964329 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:26.346237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4975d-8961-4d30-8826-13156c87f286 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:26.346410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722838f-9677-4549-8540-33cd56964329 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:26.346501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4975d-8961-4d30-8826-13156c87f286 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:26.347073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722838f-9677-4549-8540-33cd56964329 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:26.347373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722838f-9677-4549-8540-33cd56964329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:26.347732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4975d-8961-4d30-8826-13156c87f286 None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:26.348978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722838f-9677-4549-8540-33cd56964329 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:26.349352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4975d-8961-4d30-8826-13156c87f286 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:26.351334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 170/351] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:26 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:26.353979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc342941-fa39-4715-8bb4-c82072c23aa9 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:26.354444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc342941-fa39-4715-8bb4-c82072c23aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:26.354898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc342941-fa39-4715-8bb4-c82072c23aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:26.360641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 182/352] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:26.361853 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722838f-9677-4549-8540-33cd56964329 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:26.382340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-cc342941-fa39-4715-8bb4-c82072c23aa9 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:26.383655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cc342941-fa39-4715-8bb4-c82072c23aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:26.419526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:26.470017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 171/353] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:26.471540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc342941-fa39-4715-8bb4-c82072c23aa9 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:26.497691 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722838f-9677-4549-8540-33cd56964329 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:26.546051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f90336-2f8f-4726-bb02-e90319a119d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:26.547109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f90336-2f8f-4726-bb02-e90319a119d2 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:26.547595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f90336-2f8f-4726-bb02-e90319a119d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:26.548063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f90336-2f8f-4726-bb02-e90319a119d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:26.600611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:26.607678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc342941-fa39-4715-8bb4-c82072c23aa9 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:26.635399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f90336-2f8f-4726-bb02-e90319a119d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:26.637210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f90336-2f8f-4726-bb02-e90319a119d2 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:26.644577 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a080945-81dc-491f-9b86-bef5e24f2b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:26.645217 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a080945-81dc-491f-9b86-bef5e24f2b9f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:26.645497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a080945-81dc-491f-9b86-bef5e24f2b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:26.645772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a080945-81dc-491f-9b86-bef5e24f2b9f None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:26.646474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a080945-81dc-491f-9b86-bef5e24f2b9f None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:26.646779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a080945-81dc-491f-9b86-bef5e24f2b9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:26.650267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a080945-81dc-491f-9b86-bef5e24f2b9f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:26.661712 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 172/354] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:26.662927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a080945-81dc-491f-9b86-bef5e24f2b9f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:26.700598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:26.771564 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 183/355] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:26.774044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f90336-2f8f-4726-bb02-e90319a119d2 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:26.804646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a080945-81dc-491f-9b86-bef5e24f2b9f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:26.846468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4adbb-b6b0-4ece-be63-049cfa760713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:26.847108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4adbb-b6b0-4ece-be63-049cfa760713 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:26.847383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4adbb-b6b0-4ece-be63-049cfa760713 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:26.847662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4adbb-b6b0-4ece-be63-049cfa760713 None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:26.854514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4adbb-b6b0-4ece-be63-049cfa760713 None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:26.855892 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4adbb-b6b0-4ece-be63-049cfa760713 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:26.858569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 173/356] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:26 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:26.859353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4adbb-b6b0-4ece-be63-049cfa760713 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:26.882585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:26.897797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f90336-2f8f-4726-bb02-e90319a119d2 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:26.939805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd4ade-3907-4044-8b1b-f095f0828376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:26.940472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd4ade-3907-4044-8b1b-f095f0828376 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:26.940755 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd4ade-3907-4044-8b1b-f095f0828376 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:26.941142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd4ade-3907-4044-8b1b-f095f0828376 None admin] PATH_INFO: `/v3/users/4ff30a51660841b4bca2ef3de6ede040` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:26.942825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd4ade-3907-4044-8b1b-f095f0828376 None admin] RBAC: Authorizing `identity:get_user(user_id=4ff30a51660841b4bca2ef3de6ede040)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:26.945131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd4ade-3907-4044-8b1b-f095f0828376 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:26.946734 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 184/357] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:57:26 2022] GET /identity/v3/users/4ff30a51660841b4bca2ef3de6ede040 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:26.948055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ef70a-0395-4308-a0e4-78909e662205 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:26.948339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ef70a-0395-4308-a0e4-78909e662205 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:26.948623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ef70a-0395-4308-a0e4-78909e662205 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:26.951031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 185/358] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:26.951831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ef70a-0395-4308-a0e4-78909e662205 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:26.964023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:26.994300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4adbb-b6b0-4ece-be63-049cfa760713 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:27.044540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1dfac-4fcd-4289-aade-e903ec5e3135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:27.045182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1dfac-4fcd-4289-aade-e903ec5e3135 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:27.045461 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1dfac-4fcd-4289-aade-e903ec5e3135 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:27.046462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1dfac-4fcd-4289-aade-e903ec5e3135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:27.080628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ef70a-0395-4308-a0e4-78909e662205 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:27.123825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccb845-52ed-4950-8d2e-47f478d9855c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:27.124467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccb845-52ed-4950-8d2e-47f478d9855c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:27.124744 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccb845-52ed-4950-8d2e-47f478d9855c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:27.125116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccb845-52ed-4950-8d2e-47f478d9855c None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:27.126376 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccb845-52ed-4950-8d2e-47f478d9855c None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:27.127827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccb845-52ed-4950-8d2e-47f478d9855c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:27.129520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 186/359] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:26 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:27.130304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1dfac-4fcd-4289-aade-e903ec5e3135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:27.130433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccb845-52ed-4950-8d2e-47f478d9855c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:27.132694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1dfac-4fcd-4289-aade-e903ec5e3135 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:27.148652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:27.229259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:27.254161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccb845-52ed-4950-8d2e-47f478d9855c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:27.259436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 174/360] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:27.260299 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1dfac-4fcd-4289-aade-e903ec5e3135 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:27.303159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea65b6-26f0-4d45-a770-0d3cae1f4164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:27.306503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea65b6-26f0-4d45-a770-0d3cae1f4164 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:27.306503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea65b6-26f0-4d45-a770-0d3cae1f4164 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:27.306503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea65b6-26f0-4d45-a770-0d3cae1f4164 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:27.321007 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea65b6-26f0-4d45-a770-0d3cae1f4164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-71ea65b6-26f0-4d45-a770-0d3cae1f4164 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:27.325965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:27.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:27.327730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 187/361] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:57:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:27.328422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea65b6-26f0-4d45-a770-0d3cae1f4164 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:27.385246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1dfac-4fcd-4289-aade-e903ec5e3135 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:27.397814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:27.425100 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed788e73-0e52-47a6-9e47-e9da63cd26ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:27.425635 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788e73-0e52-47a6-9e47-e9da63cd26ab None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:27.425909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788e73-0e52-47a6-9e47-e9da63cd26ab None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:27.426172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788e73-0e52-47a6-9e47-e9da63cd26ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:27.426908 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed788e73-0e52-47a6-9e47-e9da63cd26ab None admin] RBAC: Adding query filter params (group.id=e8fb9a57bff44af2a0632fc095e16248, role.id=13913230583d4ce384599ac5dace7ffa, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:27.427212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed788e73-0e52-47a6-9e47-e9da63cd26ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:27.429451 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed788e73-0e52-47a6-9e47-e9da63cd26ab None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:27.435233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 175/362] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Aug 24 09:57:27 2022] GET /identity/v3/role_assignments?group.id=e8fb9a57bff44af2a0632fc095e16248&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=13913230583d4ce384599ac5dace7ffa => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:27.436645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78e1e0-7135-4610-a98c-4aa3082ee8a0 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:27.436962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78e1e0-7135-4610-a98c-4aa3082ee8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:27.437237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78e1e0-7135-4610-a98c-4aa3082ee8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:27.452704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea65b6-26f0-4d45-a770-0d3cae1f4164 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:27.460571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-fc78e1e0-7135-4610-a98c-4aa3082ee8a0 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:27.462403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-fc78e1e0-7135-4610-a98c-4aa3082ee8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:27.509896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdabd9c-6a21-4b2d-af82-443c82b08828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:27.511325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdabd9c-6a21-4b2d-af82-443c82b08828 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:27.511686 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdabd9c-6a21-4b2d-af82-443c82b08828 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:27.511969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdabd9c-6a21-4b2d-af82-443c82b08828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:27.512724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdabd9c-6a21-4b2d-af82-443c82b08828 None admin] RBAC: Adding query filter params (role.id=b9b25895899d46eea0c912f6277165cb, user.id=4ff30a51660841b4bca2ef3de6ede040, scope.system=all) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:27.513055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdabd9c-6a21-4b2d-af82-443c82b08828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:27.515175 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdabd9c-6a21-4b2d-af82-443c82b08828 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:27.517652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:27.536900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 188/363] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Aug 24 09:57:27 2022] GET /identity/v3/role_assignments?user.id=4ff30a51660841b4bca2ef3de6ede040&scope.system=all&role.id=b9b25895899d46eea0c912f6277165cb => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:27.536900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdabd9c-6a21-4b2d-af82-443c82b08828 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:27.577532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 176/364] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:27.585708 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78e1e0-7135-4610-a98c-4aa3082ee8a0 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:27.672751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdabd9c-6a21-4b2d-af82-443c82b08828 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:27.717981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152740f-6347-49b6-84fb-4e703571bb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:27.718869 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152740f-6347-49b6-84fb-4e703571bb9d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:27.719148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152740f-6347-49b6-84fb-4e703571bb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:27.719420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152740f-6347-49b6-84fb-4e703571bb9d None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:27.721066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152740f-6347-49b6-84fb-4e703571bb9d None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:27.722943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152740f-6347-49b6-84fb-4e703571bb9d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:27.725586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 189/365] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:27 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:27.726621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152740f-6347-49b6-84fb-4e703571bb9d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:27.775559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78e1e0-7135-4610-a98c-4aa3082ee8a0 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:27.785263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:27.815645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:27.830954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370375ab-bdb6-4509-8304-09d58f6384ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:27.831601 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370375ab-bdb6-4509-8304-09d58f6384ab None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:27.831930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370375ab-bdb6-4509-8304-09d58f6384ab None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:27.832208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370375ab-bdb6-4509-8304-09d58f6384ab None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:27.833400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370375ab-bdb6-4509-8304-09d58f6384ab None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:27.835238 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370375ab-bdb6-4509-8304-09d58f6384ab None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:27.837408 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 177/366] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:27 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:27.838215 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370375ab-bdb6-4509-8304-09d58f6384ab None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:27.868959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152740f-6347-49b6-84fb-4e703571bb9d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:27.916279 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9030e82-9396-473c-899a-d8de7cc55ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:27.917500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9030e82-9396-473c-899a-d8de7cc55ade None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:27.917981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9030e82-9396-473c-899a-d8de7cc55ade None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:27.918616 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9030e82-9396-473c-899a-d8de7cc55ade None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:27.920028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9030e82-9396-473c-899a-d8de7cc55ade None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:27.920572 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9030e82-9396-473c-899a-d8de7cc55ade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:27.922192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9030e82-9396-473c-899a-d8de7cc55ade None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:27.934093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 190/367] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:27.935765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9030e82-9396-473c-899a-d8de7cc55ade None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:27.971346 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370375ab-bdb6-4509-8304-09d58f6384ab None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:28.016745 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7f18e-9aeb-48bf-97a0-5f918ef0c91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:28.017633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7f18e-9aeb-48bf-97a0-5f918ef0c91f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:28.017914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7f18e-9aeb-48bf-97a0-5f918ef0c91f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:28.018226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7f18e-9aeb-48bf-97a0-5f918ef0c91f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:28.019051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7f18e-9aeb-48bf-97a0-5f918ef0c91f None admin] RBAC: Adding query filter params (group.id=e8fb9a57bff44af2a0632fc095e16248, role.id=276ae07995d14b5d8ceb8e646392c2b1, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:28.019357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7f18e-9aeb-48bf-97a0-5f918ef0c91f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:28.020796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7f18e-9aeb-48bf-97a0-5f918ef0c91f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:28.028020 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 178/368] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Aug 24 09:57:27 2022] GET /identity/v3/role_assignments?group.id=e8fb9a57bff44af2a0632fc095e16248&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=276ae07995d14b5d8ceb8e646392c2b1 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:28.029944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7f18e-9aeb-48bf-97a0-5f918ef0c91f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:28.067194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9030e82-9396-473c-899a-d8de7cc55ade None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:28.077578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:28.123350 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:28.141594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d15a41-a79a-4970-a4d1-f160b52d9551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:28.142271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15a41-a79a-4970-a4d1-f160b52d9551 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:28.142559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15a41-a79a-4970-a4d1-f160b52d9551 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:28.142843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15a41-a79a-4970-a4d1-f160b52d9551 None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:28.144836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d15a41-a79a-4970-a4d1-f160b52d9551 None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:28.146734 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d15a41-a79a-4970-a4d1-f160b52d9551 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:28.149687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 191/369] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:27 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:28.150682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d15a41-a79a-4970-a4d1-f160b52d9551 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:28.165353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7f18e-9aeb-48bf-97a0-5f918ef0c91f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:28.217037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf8539-7091-4bb3-b026-8a13a6e621ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:28.217672 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf8539-7091-4bb3-b026-8a13a6e621ff None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:28.217956 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf8539-7091-4bb3-b026-8a13a6e621ff None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:28.218235 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf8539-7091-4bb3-b026-8a13a6e621ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:28.305752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d15a41-a79a-4970-a4d1-f160b52d9551 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:28.328729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf8539-7091-4bb3-b026-8a13a6e621ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:28.331063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf8539-7091-4bb3-b026-8a13a6e621ff None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:28.347747 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590a3a-426c-461b-b10c-c4ab245f6f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:28.348385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590a3a-426c-461b-b10c-c4ab245f6f23 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:28.348671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590a3a-426c-461b-b10c-c4ab245f6f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:28.349159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590a3a-426c-461b-b10c-c4ab245f6f23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:28.364807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75590a3a-426c-461b-b10c-c4ab245f6f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-75590a3a-426c-461b-b10c-c4ab245f6f23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:28.371075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:28.371987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 192/370] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:57:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:28.374018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590a3a-426c-461b-b10c-c4ab245f6f23 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:28.386719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:28.449273 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:28.475884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 179/371] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:28.477085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf8539-7091-4bb3-b026-8a13a6e621ff None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:28.502982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590a3a-426c-461b-b10c-c4ab245f6f23 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:28.546282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135565d-5077-4075-8624-f0971f2cc9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:28.547121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135565d-5077-4075-8624-f0971f2cc9cb None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:28.547563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135565d-5077-4075-8624-f0971f2cc9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:28.548063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135565d-5077-4075-8624-f0971f2cc9cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:28.549039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135565d-5077-4075-8624-f0971f2cc9cb None admin] RBAC: Adding query filter params (group.id=e8fb9a57bff44af2a0632fc095e16248, role.id=13913230583d4ce384599ac5dace7ffa, scope.project.id=42ce709191ce4b1b961e89e218566b5b) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:28.549568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135565d-5077-4075-8624-f0971f2cc9cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:28.551237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135565d-5077-4075-8624-f0971f2cc9cb None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:28.561128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 193/372] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Aug 24 09:57:28 2022] GET /identity/v3/role_assignments?group.id=e8fb9a57bff44af2a0632fc095e16248&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=13913230583d4ce384599ac5dace7ffa => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:28.562000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135565d-5077-4075-8624-f0971f2cc9cb None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:28.638446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf8539-7091-4bb3-b026-8a13a6e621ff None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:28.681861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135565d-5077-4075-8624-f0971f2cc9cb None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:28.706276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb2aa1-cb75-4398-af43-e196bd59194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:28.706902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2aa1-cb75-4398-af43-e196bd59194c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:28.707181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2aa1-cb75-4398-af43-e196bd59194c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:28.707356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:28.707458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2aa1-cb75-4398-af43-e196bd59194c None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:28.708113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb2aa1-cb75-4398-af43-e196bd59194c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:28.708412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb2aa1-cb75-4398-af43-e196bd59194c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:28.710478 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb2aa1-cb75-4398-af43-e196bd59194c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:28.713163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:28.724564 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf17f4-3b95-46c3-a246-c820876b4c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:28.725897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf17f4-3b95-46c3-a246-c820876b4c34 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:28.726066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 180/373] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:28.726583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf17f4-3b95-46c3-a246-c820876b4c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:28.731969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf17f4-3b95-46c3-a246-c820876b4c34 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:28.731969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf17f4-3b95-46c3-a246-c820876b4c34 None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:28.731969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf17f4-3b95-46c3-a246-c820876b4c34 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:28.732194 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb2aa1-cb75-4398-af43-e196bd59194c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:28.732263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 194/374] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:28 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:28.737744 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf17f4-3b95-46c3-a246-c820876b4c34 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:28.888246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf17f4-3b95-46c3-a246-c820876b4c34 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:28.929944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a66b03-5dd0-4698-a893-77f50718eb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:28.930587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66b03-5dd0-4698-a893-77f50718eb3b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:28.930868 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66b03-5dd0-4698-a893-77f50718eb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:28.931141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66b03-5dd0-4698-a893-77f50718eb3b None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:28.932265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a66b03-5dd0-4698-a893-77f50718eb3b None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:28.934874 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a66b03-5dd0-4698-a893-77f50718eb3b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:28.936993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 195/375] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:28 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:28.939047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb2aa1-cb75-4398-af43-e196bd59194c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:28.944568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a66b03-5dd0-4698-a893-77f50718eb3b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:29.010385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:29.049632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccf644-5461-45cb-8d03-bb1a78c5986c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:29.050263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccf644-5461-45cb-8d03-bb1a78c5986c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:29.050544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccf644-5461-45cb-8d03-bb1a78c5986c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:29.050814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccf644-5461-45cb-8d03-bb1a78c5986c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:29.072038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a66b03-5dd0-4698-a893-77f50718eb3b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:29.113302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8b52b-6931-4d16-b3f3-5be04d8a751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:29.117437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8b52b-6931-4d16-b3f3-5be04d8a751e None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:29.118079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8b52b-6931-4d16-b3f3-5be04d8a751e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:29.119215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8b52b-6931-4d16-b3f3-5be04d8a751e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:29.120128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8b52b-6931-4d16-b3f3-5be04d8a751e None admin] RBAC: Adding query filter params (group.id=e8fb9a57bff44af2a0632fc095e16248, role.id=276ae07995d14b5d8ceb8e646392c2b1, scope.project.id=42ce709191ce4b1b961e89e218566b5b) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:29.121542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8b52b-6931-4d16-b3f3-5be04d8a751e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:29.125738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8b52b-6931-4d16-b3f3-5be04d8a751e None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:29.140958 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 196/376] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Aug 24 09:57:28 2022] GET /identity/v3/role_assignments?group.id=e8fb9a57bff44af2a0632fc095e16248&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=276ae07995d14b5d8ceb8e646392c2b1 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:29.181156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:29.185630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ccf644-5461-45cb-8d03-bb1a78c5986c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:29.187544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ccf644-5461-45cb-8d03-bb1a78c5986c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:29.313771 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 181/377] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:57:29.320108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8b52b-6931-4d16-b3f3-5be04d8a751e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:29.473813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8b52b-6931-4d16-b3f3-5be04d8a751e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:29.497368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:29.523135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e1b61-b1a7-4968-bfb6-20b0895f5a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:29.524111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e1b61-b1a7-4968-bfb6-20b0895f5a53 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:29.524578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e1b61-b1a7-4968-bfb6-20b0895f5a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:29.525130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e1b61-b1a7-4968-bfb6-20b0895f5a53 None admin] PATH_INFO: `/v3/projects/85765b8a176a42c8b2b1e092bbcf9c4f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:29.526537 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e1b61-b1a7-4968-bfb6-20b0895f5a53 None admin] RBAC: Authorizing `identity:get_project(project_id=85765b8a176a42c8b2b1e092bbcf9c4f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:29.528152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e1b61-b1a7-4968-bfb6-20b0895f5a53 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:29.530349 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 197/378] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:29 2022] GET /identity/v3/projects/85765b8a176a42c8b2b1e092bbcf9c4f => generated 344 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:29.535690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccf644-5461-45cb-8d03-bb1a78c5986c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:29.655356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccf644-5461-45cb-8d03-bb1a78c5986c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:29.660892 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:29.708344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c2416-c844-4612-908b-8feb32415409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:29.709412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c2416-c844-4612-908b-8feb32415409 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:29.710399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c2416-c844-4612-908b-8feb32415409 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:29.710894 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c2416-c844-4612-908b-8feb32415409 None admin] PATH_INFO: `/v3/groups/06f83eb2b41b4ccabcc238999d04b881` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:29.716360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c2416-c844-4612-908b-8feb32415409 None admin] RBAC: Authorizing `identity:get_group(group_id=06f83eb2b41b4ccabcc238999d04b881)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:29.718066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c2416-c844-4612-908b-8feb32415409 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:29.719952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 182/379] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:29 2022] GET /identity/v3/groups/06f83eb2b41b4ccabcc238999d04b881 => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:29.726225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e1b61-b1a7-4968-bfb6-20b0895f5a53 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:29.797156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbb1d5-fb4a-4977-8a71-674d67b414d7 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:29.797799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbb1d5-fb4a-4977-8a71-674d67b414d7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:29.798345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbb1d5-fb4a-4977-8a71-674d67b414d7 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:29.799935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 183/380] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:57:29.806034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7b9ea-d52f-4c6a-a0e1-0f73a56e4b1d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:29.806722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7b9ea-d52f-4c6a-a0e1-0f73a56e4b1d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:29.807219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7b9ea-d52f-4c6a-a0e1-0f73a56e4b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:29.838166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b0d7b9ea-d52f-4c6a-a0e1-0f73a56e4b1d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:29.839780 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b0d7b9ea-d52f-4c6a-a0e1-0f73a56e4b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:29.852963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e1b61-b1a7-4968-bfb6-20b0895f5a53 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:29.888268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd137f-8f69-4962-89c0-9c7800429ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:29.888894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd137f-8f69-4962-89c0-9c7800429ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:29.889187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd137f-8f69-4962-89c0-9c7800429ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:29.889464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd137f-8f69-4962-89c0-9c7800429ee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:29.890189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd137f-8f69-4962-89c0-9c7800429ee5 None admin] RBAC: Adding query filter params (group.id=06f83eb2b41b4ccabcc238999d04b881, role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=85765b8a176a42c8b2b1e092bbcf9c4f) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:29.890498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd137f-8f69-4962-89c0-9c7800429ee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:29.891933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd137f-8f69-4962-89c0-9c7800429ee5 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:29.898745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 198/381] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Aug 24 09:57:29 2022] GET /identity/v3/role_assignments?group.id=06f83eb2b41b4ccabcc238999d04b881&scope.project.id=85765b8a176a42c8b2b1e092bbcf9c4f&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:29.988390 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 184/382] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:30.262862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6ec6b-2f4d-44c5-8794-b8982980861b None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:30.263218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6ec6b-2f4d-44c5-8794-b8982980861b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:30.263333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6ec6b-2f4d-44c5-8794-b8982980861b None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:30.264707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 199/383] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:30.270325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914e3ce-8bdb-4cd5-91bd-7d0f7d5c6085 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:30.270617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914e3ce-8bdb-4cd5-91bd-7d0f7d5c6085 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:30.270899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914e3ce-8bdb-4cd5-91bd-7d0f7d5c6085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:30.291333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3914e3ce-8bdb-4cd5-91bd-7d0f7d5c6085 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:30.292531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3914e3ce-8bdb-4cd5-91bd-7d0f7d5c6085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:30.324598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6ec6b-2f4d-44c5-8794-b8982980861b None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:30.412222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:30.424622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 185/384] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:30.478571 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6ec6b-2f4d-44c5-8794-b8982980861b None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:30.514158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9db365-4bae-4d04-8ec6-d52abacfdb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:30.514709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9db365-4bae-4d04-8ec6-d52abacfdb8c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:30.515045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9db365-4bae-4d04-8ec6-d52abacfdb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:30.515318 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9db365-4bae-4d04-8ec6-d52abacfdb8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:30.526763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9db365-4bae-4d04-8ec6-d52abacfdb8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-1d9db365-4bae-4d04-8ec6-d52abacfdb8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:30.532244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:30.533162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:30.533974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 200/385] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:30.542621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914e3ce-8bdb-4cd5-91bd-7d0f7d5c6085 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:30.579952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:30.672258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914e3ce-8bdb-4cd5-91bd-7d0f7d5c6085 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:30.709315 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ff1b2-286a-4c34-b6cb-c97caf450290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:30.709862 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ff1b2-286a-4c34-b6cb-c97caf450290 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:30.710136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ff1b2-286a-4c34-b6cb-c97caf450290 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:30.710408 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ff1b2-286a-4c34-b6cb-c97caf450290 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:30.711071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ff1b2-286a-4c34-b6cb-c97caf450290 None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:30.711370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ff1b2-286a-4c34-b6cb-c97caf450290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:30.712762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ff1b2-286a-4c34-b6cb-c97caf450290 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:30.721660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 186/386] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:30.729994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9db365-4bae-4d04-8ec6-d52abacfdb8c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:30.800899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ff1b2-286a-4c34-b6cb-c97caf450290 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:30.905811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:30.915471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9db365-4bae-4d04-8ec6-d52abacfdb8c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:30.925287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ff1b2-286a-4c34-b6cb-c97caf450290 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:30.935127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:30.982681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc532f-b877-4f48-ab40-5063e991ff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:30.983734 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc532f-b877-4f48-ab40-5063e991ff9b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:30.984258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc532f-b877-4f48-ab40-5063e991ff9b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:30.984978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc532f-b877-4f48-ab40-5063e991ff9b None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:30.988729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc532f-b877-4f48-ab40-5063e991ff9b None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:30.991073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc532f-b877-4f48-ab40-5063e991ff9b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:30.994726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 201/387] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:30 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:30.998306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8e8f8-3b4a-483d-9671-65a2e63242ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:30.999287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8e8f8-3b4a-483d-9671-65a2e63242ab None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:31.000303 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc532f-b877-4f48-ab40-5063e991ff9b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:31.001149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8e8f8-3b4a-483d-9671-65a2e63242ab None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:31.001985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8e8f8-3b4a-483d-9671-65a2e63242ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:31.019079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8e8f8-3b4a-483d-9671-65a2e63242ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-a4f8e8f8-3b4a-483d-9671-65a2e63242ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:31.033252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:31.034212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:31.034996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 187/388] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:57:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:57:31.042773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8e8f8-3b4a-483d-9671-65a2e63242ab None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:31.120449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc532f-b877-4f48-ab40-5063e991ff9b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:31.157865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76694d58-1ea7-45a4-aa26-acafb73011d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:31.158845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76694d58-1ea7-45a4-aa26-acafb73011d5 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:31.159423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76694d58-1ea7-45a4-aa26-acafb73011d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:31.159924 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76694d58-1ea7-45a4-aa26-acafb73011d5 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:31.161313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76694d58-1ea7-45a4-aa26-acafb73011d5 None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:31.162994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76694d58-1ea7-45a4-aa26-acafb73011d5 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:31.165389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 202/389] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:30 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:31.166822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9f50-0628-4135-8e80-c7f1830b977a None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:31.167294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9f50-0628-4135-8e80-c7f1830b977a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:31.167575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9f50-0628-4135-8e80-c7f1830b977a None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:31.168892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 203/390] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:31.170759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b9f50-0628-4135-8e80-c7f1830b977a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:31.220146 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8e8f8-3b4a-483d-9671-65a2e63242ab None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:31.223092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:31.267350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fedb4-0b16-4c57-8492-5bc817f021b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:31.267999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fedb4-0b16-4c57-8492-5bc817f021b0 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:31.268278 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fedb4-0b16-4c57-8492-5bc817f021b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:31.268552 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fedb4-0b16-4c57-8492-5bc817f021b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:31.269259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fedb4-0b16-4c57-8492-5bc817f021b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:31.269560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fedb4-0b16-4c57-8492-5bc817f021b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:31.271486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fedb4-0b16-4c57-8492-5bc817f021b0 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:31.289180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 188/391] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:31.289180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e167f-bdc4-4df5-8ec2-f5e36f2a0f74 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:31.289180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e167f-bdc4-4df5-8ec2-f5e36f2a0f74 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:31.289180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e167f-bdc4-4df5-8ec2-f5e36f2a0f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:31.291623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b9f50-0628-4135-8e80-c7f1830b977a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:31.301466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:31.309506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-7e7e167f-bdc4-4df5-8ec2-f5e36f2a0f74 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:31.313876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7e7e167f-bdc4-4df5-8ec2-f5e36f2a0f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:31.338022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a2af-74b5-4f33-b059-19f0503f0d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:31.339927 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a2af-74b5-4f33-b059-19f0503f0d61 None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:31.340702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a2af-74b5-4f33-b059-19f0503f0d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:31.341223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a2af-74b5-4f33-b059-19f0503f0d61 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/groups/e8fb9a57bff44af2a0632fc095e16248/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:31.344628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4a2af-74b5-4f33-b059-19f0503f0d61 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4bdbfd07cd94eeea5087a9d39f83369, group_id=e8fb9a57bff44af2a0632fc095e16248, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:31.346858 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4a2af-74b5-4f33-b059-19f0503f0d61 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:31.396784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 189/392] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:31.398478 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e167f-bdc4-4df5-8ec2-f5e36f2a0f74 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:31.444984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 204/393] 10.222.0.30 () {58 vars in 1442 bytes} [Wed Aug 24 09:57:31 2022] PUT /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/groups/e8fb9a57bff44af2a0632fc095e16248/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:57:31.525902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e167f-bdc4-4df5-8ec2-f5e36f2a0f74 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:31.575239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45c28-be27-408f-af95-565a32d2ce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:31.576271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45c28-be27-408f-af95-565a32d2ce57 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:31.576753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45c28-be27-408f-af95-565a32d2ce57 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:31.577276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45c28-be27-408f-af95-565a32d2ce57 None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:31.584888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d45c28-be27-408f-af95-565a32d2ce57 None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:31.587121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d45c28-be27-408f-af95-565a32d2ce57 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:31.589096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 190/394] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:31 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:31.595067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a2af-74b5-4f33-b059-19f0503f0d61 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:31.719815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45c28-be27-408f-af95-565a32d2ce57 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:31.725695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:31.736414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a2af-74b5-4f33-b059-19f0503f0d61 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:31.739457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:31.779204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaab005-7bc0-4767-b5a8-9e56fb47ea54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:31.781249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab005-7bc0-4767-b5a8-9e56fb47ea54 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:31.781249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab005-7bc0-4767-b5a8-9e56fb47ea54 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:31.781249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab005-7bc0-4767-b5a8-9e56fb47ea54 None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:31.783375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaab005-7bc0-4767-b5a8-9e56fb47ea54 None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:31.785875 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaab005-7bc0-4767-b5a8-9e56fb47ea54 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:31.790376 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 205/395] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:31 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:31.794488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaab005-7bc0-4767-b5a8-9e56fb47ea54 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:31.927990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45c28-be27-408f-af95-565a32d2ce57 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:31.983961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589758a0-d94a-4f69-a14b-9ea898417c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:31.984523 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589758a0-d94a-4f69-a14b-9ea898417c61 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:31.984799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589758a0-d94a-4f69-a14b-9ea898417c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:31.985111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589758a0-d94a-4f69-a14b-9ea898417c61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:31.985699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaab005-7bc0-4767-b5a8-9e56fb47ea54 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:32.000407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589758a0-d94a-4f69-a14b-9ea898417c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-589758a0-d94a-4f69-a14b-9ea898417c61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:32.009282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:32.010324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:32.011120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:32.012952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 191/396] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:57:32.014246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa988b97-87f3-40d4-a869-96621669fba5 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:32.014515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa988b97-87f3-40d4-a869-96621669fba5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:32.014785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa988b97-87f3-40d4-a869-96621669fba5 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:32.017640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 192/397] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:32.018403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa988b97-87f3-40d4-a869-96621669fba5 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:32.035511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c39c5-c0ea-4be6-bd0d-c20a94f6c1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:32.036742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c39c5-c0ea-4be6-bd0d-c20a94f6c1df None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:32.037032 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c39c5-c0ea-4be6-bd0d-c20a94f6c1df None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:32.037977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c39c5-c0ea-4be6-bd0d-c20a94f6c1df None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/groups/e8fb9a57bff44af2a0632fc095e16248/roles/276ae07995d14b5d8ceb8e646392c2b1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:32.040763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c39c5-c0ea-4be6-bd0d-c20a94f6c1df None admin] RBAC: Authorizing `identity:create_grant(project_id=c4bdbfd07cd94eeea5087a9d39f83369, group_id=e8fb9a57bff44af2a0632fc095e16248, role_id=276ae07995d14b5d8ceb8e646392c2b1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:32.043041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c39c5-c0ea-4be6-bd0d-c20a94f6c1df None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:32.095789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:32.137566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 206/398] 10.222.0.30 () {58 vars in 1442 bytes} [Wed Aug 24 09:57:31 2022] PUT /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369/groups/e8fb9a57bff44af2a0632fc095e16248/roles/276ae07995d14b5d8ceb8e646392c2b1 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:32.139452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0060-f756-4099-9760-0f13159f7686 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:32.139943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0060-f756-4099-9760-0f13159f7686 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:32.140431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0060-f756-4099-9760-0f13159f7686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:32.164816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-098a0060-f756-4099-9760-0f13159f7686 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:32.166683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-098a0060-f756-4099-9760-0f13159f7686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:32.179780 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa988b97-87f3-40d4-a869-96621669fba5 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:32.222536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:32.261764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69cfae-2489-4c16-bf05-b61400477929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:32.262289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69cfae-2489-4c16-bf05-b61400477929 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:32.262563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69cfae-2489-4c16-bf05-b61400477929 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:32.262832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69cfae-2489-4c16-bf05-b61400477929 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:32.265080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69cfae-2489-4c16-bf05-b61400477929 None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:32.265080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69cfae-2489-4c16-bf05-b61400477929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:32.266172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69cfae-2489-4c16-bf05-b61400477929 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:32.293195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 193/399] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:32.309580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69cfae-2489-4c16-bf05-b61400477929 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:32.325274 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 207/400] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:32.327568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2f425-85cf-4996-97e0-8c243d557d03 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:32.327848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2f425-85cf-4996-97e0-8c243d557d03 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:32.328121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2f425-85cf-4996-97e0-8c243d557d03 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:32.330220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 208/401] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:32.347246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32ff3a-ca47-49f6-b93d-2c662769d7fb None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:32.347848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32ff3a-ca47-49f6-b93d-2c662769d7fb None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:32.347848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32ff3a-ca47-49f6-b93d-2c662769d7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:32.371472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-da32ff3a-ca47-49f6-b93d-2c662769d7fb None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:32.372675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-da32ff3a-ca47-49f6-b93d-2c662769d7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:32.438747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69cfae-2489-4c16-bf05-b61400477929 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:32.470644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:32.475701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 209/402] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:32.481650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d85c77-b021-4297-8506-57f2ebfa41d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:32.482373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d85c77-b021-4297-8506-57f2ebfa41d0 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:32.482741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d85c77-b021-4297-8506-57f2ebfa41d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:32.483083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d85c77-b021-4297-8506-57f2ebfa41d0 None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:32.484289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d85c77-b021-4297-8506-57f2ebfa41d0 None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:32.485888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d85c77-b021-4297-8506-57f2ebfa41d0 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:32.487849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 194/403] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:32 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:32.494071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32ff3a-ca47-49f6-b93d-2c662769d7fb None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:32.639262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:32.647861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32ff3a-ca47-49f6-b93d-2c662769d7fb None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:32.674042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d85c77-b021-4297-8506-57f2ebfa41d0 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:32.699845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00936c-e88f-462f-b925-1901a1e6b025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:32.700489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00936c-e88f-462f-b925-1901a1e6b025 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:32.700766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00936c-e88f-462f-b925-1901a1e6b025 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:32.701061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00936c-e88f-462f-b925-1901a1e6b025 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:32.703494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f00936c-e88f-462f-b925-1901a1e6b025 None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:32.705865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f00936c-e88f-462f-b925-1901a1e6b025 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:32.715811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 210/404] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:32 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:32.717436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00936c-e88f-462f-b925-1901a1e6b025 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:32.881607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d85c77-b021-4297-8506-57f2ebfa41d0 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:32.891135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00936c-e88f-462f-b925-1901a1e6b025 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:32.923846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a851545-0891-4e9e-95da-b81a6dcbb399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:32.924641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a851545-0891-4e9e-95da-b81a6dcbb399 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:32.924951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a851545-0891-4e9e-95da-b81a6dcbb399 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:32.925227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a851545-0891-4e9e-95da-b81a6dcbb399 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:32.934700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed2ac1-3e89-4a17-85f8-5620931c9c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:32.935714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed2ac1-3e89-4a17-85f8-5620931c9c76 None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:32.936061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed2ac1-3e89-4a17-85f8-5620931c9c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:32.936407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed2ac1-3e89-4a17-85f8-5620931c9c76 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b/groups/e8fb9a57bff44af2a0632fc095e16248/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:32.937735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a851545-0891-4e9e-95da-b81a6dcbb399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:32.938796 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed2ac1-3e89-4a17-85f8-5620931c9c76 None admin] RBAC: Authorizing `identity:create_grant(project_id=42ce709191ce4b1b961e89e218566b5b, group_id=e8fb9a57bff44af2a0632fc095e16248, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:32.940373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed2ac1-3e89-4a17-85f8-5620931c9c76 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-5a851545-0891-4e9e-95da-b81a6dcbb399 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:32.943354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:32.945649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 195/405] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:57:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:57:32.947593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a851545-0891-4e9e-95da-b81a6dcbb399 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:33.010547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:33.060199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a851545-0891-4e9e-95da-b81a6dcbb399 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:33.089928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 211/406] 10.222.0.30 () {58 vars in 1442 bytes} [Wed Aug 24 09:57:32 2022] PUT /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b/groups/e8fb9a57bff44af2a0632fc095e16248/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:57:33.092143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed2ac1-3e89-4a17-85f8-5620931c9c76 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:33.102512 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f18dc6-9671-476b-a749-f62242ef11a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:33.103198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f18dc6-9671-476b-a749-f62242ef11a5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.103483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f18dc6-9671-476b-a749-f62242ef11a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.103757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f18dc6-9671-476b-a749-f62242ef11a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.116593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f18dc6-9671-476b-a749-f62242ef11a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-e0f18dc6-9671-476b-a749-f62242ef11a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:33.123578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:33.124547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:33.125371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:33.128759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 196/407] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:57:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:33.136768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f18dc6-9671-476b-a749-f62242ef11a5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:33.163519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:33.249709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f18dc6-9671-476b-a749-f62242ef11a5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:33.273944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:33.304803 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69c42d-9dcd-4a09-9071-de0a831b28d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:33.306768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69c42d-9dcd-4a09-9071-de0a831b28d6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.307937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69c42d-9dcd-4a09-9071-de0a831b28d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.308471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69c42d-9dcd-4a09-9071-de0a831b28d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.310940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69c42d-9dcd-4a09-9071-de0a831b28d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:33.311456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69c42d-9dcd-4a09-9071-de0a831b28d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:33.314284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69c42d-9dcd-4a09-9071-de0a831b28d6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:33.320067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed2ac1-3e89-4a17-85f8-5620931c9c76 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:33.337437 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 197/408] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:33.345686 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69c42d-9dcd-4a09-9071-de0a831b28d6 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:33.396499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddf4c9-741b-4e2a-9a9a-9474097a85e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:33.397960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddf4c9-741b-4e2a-9a9a-9474097a85e6 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.398445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddf4c9-741b-4e2a-9a9a-9474097a85e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.398912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddf4c9-741b-4e2a-9a9a-9474097a85e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.400081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddf4c9-741b-4e2a-9a9a-9474097a85e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:33.400602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddf4c9-741b-4e2a-9a9a-9474097a85e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:33.402330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddf4c9-741b-4e2a-9a9a-9474097a85e6 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:33.417338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 212/409] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:33.434126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddf4c9-741b-4e2a-9a9a-9474097a85e6 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:33.554350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69c42d-9dcd-4a09-9071-de0a831b28d6 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:33.593242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:33.627494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f6682-8e28-4bde-88e0-c8e0c0290e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:33.628139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f6682-8e28-4bde-88e0-c8e0c0290e5a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.628411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f6682-8e28-4bde-88e0-c8e0c0290e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.628680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f6682-8e28-4bde-88e0-c8e0c0290e5a None admin] PATH_INFO: `/v3/groups/06f83eb2b41b4ccabcc238999d04b881` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.629681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddf4c9-741b-4e2a-9a9a-9474097a85e6 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:33.631778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f6682-8e28-4bde-88e0-c8e0c0290e5a None admin] RBAC: Authorizing `identity:get_group(group_id=06f83eb2b41b4ccabcc238999d04b881)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:33.633879 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f6682-8e28-4bde-88e0-c8e0c0290e5a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:33.636351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 198/410] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:33 2022] GET /identity/v3/groups/06f83eb2b41b4ccabcc238999d04b881 => generated 229 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:33.642374 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f6682-8e28-4bde-88e0-c8e0c0290e5a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:33.705731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3969c-6200-4b68-9362-074d1bf0638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:33.706286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3969c-6200-4b68-9362-074d1bf0638a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.706564 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3969c-6200-4b68-9362-074d1bf0638a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.706835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3969c-6200-4b68-9362-074d1bf0638a None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.708749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3969c-6200-4b68-9362-074d1bf0638a None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:33.714947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3969c-6200-4b68-9362-074d1bf0638a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:33.723046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 213/411] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:33 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:33.727125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3969c-6200-4b68-9362-074d1bf0638a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:33.736578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:33.830494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f6682-8e28-4bde-88e0-c8e0c0290e5a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:33.888838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3969c-6200-4b68-9362-074d1bf0638a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:33.914747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62da7af-a824-440b-b0aa-44ae5f47afef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:33.915778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da7af-a824-440b-b0aa-44ae5f47afef None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.916259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da7af-a824-440b-b0aa-44ae5f47afef None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.917085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da7af-a824-440b-b0aa-44ae5f47afef None admin] PATH_INFO: `/v3/projects/85765b8a176a42c8b2b1e092bbcf9c4f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.919540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62da7af-a824-440b-b0aa-44ae5f47afef None admin] RBAC: Authorizing `identity:get_project(project_id=85765b8a176a42c8b2b1e092bbcf9c4f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:33.922778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62da7af-a824-440b-b0aa-44ae5f47afef None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:33.925890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 199/412] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:33 2022] GET /identity/v3/projects/85765b8a176a42c8b2b1e092bbcf9c4f => generated 344 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:33.927520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eb3ef-c2aa-40da-813e-68fd22672f02 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.927793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eb3ef-c2aa-40da-813e-68fd22672f02 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.928077 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eb3ef-c2aa-40da-813e-68fd22672f02 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.929860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 200/413] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:33.931085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006eb3ef-c2aa-40da-813e-68fd22672f02 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:33.939123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12397be-2d5a-4d28-bc05-ef65e0f0591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:33.940023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12397be-2d5a-4d28-bc05-ef65e0f0591f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.940479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12397be-2d5a-4d28-bc05-ef65e0f0591f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.940984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12397be-2d5a-4d28-bc05-ef65e0f0591f None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.942324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12397be-2d5a-4d28-bc05-ef65e0f0591f None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:33.944457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12397be-2d5a-4d28-bc05-ef65e0f0591f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:33.946834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 214/414] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:33 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:33.952512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15759e15-13fb-4e30-810d-38305b1f5e3c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.953550 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15759e15-13fb-4e30-810d-38305b1f5e3c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.954284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15759e15-13fb-4e30-810d-38305b1f5e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.994352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-15759e15-13fb-4e30-810d-38305b1f5e3c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:33.997728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-15759e15-13fb-4e30-810d-38305b1f5e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:34.020092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:34.081629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006eb3ef-c2aa-40da-813e-68fd22672f02 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:34.087325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 215/415] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:34.088458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15759e15-13fb-4e30-810d-38305b1f5e3c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:34.110250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:34.149417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04893f7c-bc94-4ec2-a105-166fa459c053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:34.150185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04893f7c-bc94-4ec2-a105-166fa459c053 None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:34.150537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04893f7c-bc94-4ec2-a105-166fa459c053 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:34.150903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04893f7c-bc94-4ec2-a105-166fa459c053 None admin] PATH_INFO: `/v3/projects/85765b8a176a42c8b2b1e092bbcf9c4f/groups/06f83eb2b41b4ccabcc238999d04b881/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:34.157308 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04893f7c-bc94-4ec2-a105-166fa459c053 None admin] RBAC: Authorizing `identity:create_grant(project_id=85765b8a176a42c8b2b1e092bbcf9c4f, group_id=06f83eb2b41b4ccabcc238999d04b881, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:34.165025 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04893f7c-bc94-4ec2-a105-166fa459c053 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:34.200713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15759e15-13fb-4e30-810d-38305b1f5e3c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:34.238421 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df479397-d4b1-4aa4-b1ee-eaa94dc6c313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:34.239409 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df479397-d4b1-4aa4-b1ee-eaa94dc6c313 None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:34.239865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df479397-d4b1-4aa4-b1ee-eaa94dc6c313 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:34.240844 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df479397-d4b1-4aa4-b1ee-eaa94dc6c313 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b/groups/e8fb9a57bff44af2a0632fc095e16248/roles/276ae07995d14b5d8ceb8e646392c2b1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:34.243266 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df479397-d4b1-4aa4-b1ee-eaa94dc6c313 None admin] RBAC: Authorizing `identity:create_grant(project_id=42ce709191ce4b1b961e89e218566b5b, group_id=e8fb9a57bff44af2a0632fc095e16248, role_id=276ae07995d14b5d8ceb8e646392c2b1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:34.244976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df479397-d4b1-4aa4-b1ee-eaa94dc6c313 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:34.258857 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 201/416] 10.222.0.30 () {58 vars in 1442 bytes} [Wed Aug 24 09:57:33 2022] PUT /identity/v3/projects/85765b8a176a42c8b2b1e092bbcf9c4f/groups/06f83eb2b41b4ccabcc238999d04b881/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:34.350382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 216/417] 10.222.0.30 () {58 vars in 1442 bytes} [Wed Aug 24 09:57:34 2022] PUT /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b/groups/e8fb9a57bff44af2a0632fc095e16248/roles/276ae07995d14b5d8ceb8e646392c2b1 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:34.415669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd1c06-9e7e-4576-9e46-c5309f3cd696 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:34.416362 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd1c06-9e7e-4576-9e46-c5309f3cd696 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:34.416837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd1c06-9e7e-4576-9e46-c5309f3cd696 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:34.418911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 202/418] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:34.425087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b5e41-f546-4bbe-85d7-6743b8913082 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:34.426060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b5e41-f546-4bbe-85d7-6743b8913082 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:34.426562 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b5e41-f546-4bbe-85d7-6743b8913082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:34.451079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-f02b5e41-f546-4bbe-85d7-6743b8913082 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:34.452671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f02b5e41-f546-4bbe-85d7-6743b8913082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:34.455505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:34.461038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd1c06-9e7e-4576-9e46-c5309f3cd696 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:34.535770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 217/419] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:34.598848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd1c06-9e7e-4576-9e46-c5309f3cd696 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:34.649297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9c0ef-c5ed-4259-8337-f14525933cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:34.649938 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9c0ef-c5ed-4259-8337-f14525933cad None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:34.650223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9c0ef-c5ed-4259-8337-f14525933cad None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:34.650559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9c0ef-c5ed-4259-8337-f14525933cad None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:34.661293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9c0ef-c5ed-4259-8337-f14525933cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-38a9c0ef-c5ed-4259-8337-f14525933cad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:34.667089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:34.667960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:34.668772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 203/420] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:57:34.675421 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b5e41-f546-4bbe-85d7-6743b8913082 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:34.831835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b5e41-f546-4bbe-85d7-6743b8913082 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:34.839384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9c0ef-c5ed-4259-8337-f14525933cad None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:34.891589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb5fb5-50c1-4d27-9610-e4cfba366968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:34.892210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb5fb5-50c1-4d27-9610-e4cfba366968 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:34.892484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb5fb5-50c1-4d27-9610-e4cfba366968 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:34.892739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb5fb5-50c1-4d27-9610-e4cfba366968 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:34.893436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb5fb5-50c1-4d27-9610-e4cfba366968 None admin] RBAC: Adding query filter params (name=member) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:34.893732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb5fb5-50c1-4d27-9610-e4cfba366968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:34.895109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb5fb5-50c1-4d27-9610-e4cfba366968 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:34.899205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:34.902482 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:34.908803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 218/421] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:34.915584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb5fb5-50c1-4d27-9610-e4cfba366968 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:35.038767 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9c0ef-c5ed-4259-8337-f14525933cad None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:35.085519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb5fb5-50c1-4d27-9610-e4cfba366968 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:35.122439 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75652123-e0bc-4783-8003-5b68b5441f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:35.123047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75652123-e0bc-4783-8003-5b68b5441f1d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:35.123324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75652123-e0bc-4783-8003-5b68b5441f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:35.123594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75652123-e0bc-4783-8003-5b68b5441f1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:35.134246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f821c-0841-45a0-9720-c5dd6182cc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:35.134898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f821c-0841-45a0-9720-c5dd6182cc73 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:35.135180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f821c-0841-45a0-9720-c5dd6182cc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:35.135455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f821c-0841-45a0-9720-c5dd6182cc73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:35.165892 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f821c-0841-45a0-9720-c5dd6182cc73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-9d1f821c-0841-45a0-9720-c5dd6182cc73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:35.170695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:35.171605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:35.172388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 204/422] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:57:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:57:35.179435 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f821c-0841-45a0-9720-c5dd6182cc73 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:35.194376 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:35.199049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75652123-e0bc-4783-8003-5b68b5441f1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:35.200504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75652123-e0bc-4783-8003-5b68b5441f1d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:35.381822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:35.386018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 219/423] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:57:35.392668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75652123-e0bc-4783-8003-5b68b5441f1d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:35.396391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f821c-0841-45a0-9720-c5dd6182cc73 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:35.435491 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e374e0-2773-4680-a4ce-787cfff919a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:35.436105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e374e0-2773-4680-a4ce-787cfff919a5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:35.436382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e374e0-2773-4680-a4ce-787cfff919a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:35.436654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e374e0-2773-4680-a4ce-787cfff919a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:35.437344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e374e0-2773-4680-a4ce-787cfff919a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:35.437643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e374e0-2773-4680-a4ce-787cfff919a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:35.439380 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e374e0-2773-4680-a4ce-787cfff919a5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:35.449885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 205/424] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:35.455445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e374e0-2773-4680-a4ce-787cfff919a5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:35.569493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:35.573743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75652123-e0bc-4783-8003-5b68b5441f1d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:35.610948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11b0f7-47c6-4268-8449-4dda75b3de0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:35.611683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11b0f7-47c6-4268-8449-4dda75b3de0a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:35.612038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11b0f7-47c6-4268-8449-4dda75b3de0a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:35.612405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11b0f7-47c6-4268-8449-4dda75b3de0a None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:35.613625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11b0f7-47c6-4268-8449-4dda75b3de0a None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:35.615362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11b0f7-47c6-4268-8449-4dda75b3de0a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:35.615944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e374e0-2773-4680-a4ce-787cfff919a5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:35.618130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 220/425] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:35 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:35.620235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7def3-50ba-4fbb-b3a9-568bed5023cd None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:35.620235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7def3-50ba-4fbb-b3a9-568bed5023cd None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:35.620648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7def3-50ba-4fbb-b3a9-568bed5023cd None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:35.626346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 221/426] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:35.626346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7def3-50ba-4fbb-b3a9-568bed5023cd None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:35.674155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72e89e-6a4d-49bb-9fea-9ebc79584204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:35.674687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72e89e-6a4d-49bb-9fea-9ebc79584204 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:35.674974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72e89e-6a4d-49bb-9fea-9ebc79584204 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:35.675251 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72e89e-6a4d-49bb-9fea-9ebc79584204 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:35.712214 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:35.766192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7def3-50ba-4fbb-b3a9-568bed5023cd None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:35.775278 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72e89e-6a4d-49bb-9fea-9ebc79584204 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:35.777566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72e89e-6a4d-49bb-9fea-9ebc79584204 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:35.808381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52321a-1483-4260-8ac4-eb130a6f878b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:35.809169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52321a-1483-4260-8ac4-eb130a6f878b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:35.810163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52321a-1483-4260-8ac4-eb130a6f878b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:35.810506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52321a-1483-4260-8ac4-eb130a6f878b None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:35.812035 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52321a-1483-4260-8ac4-eb130a6f878b None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:35.814375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52321a-1483-4260-8ac4-eb130a6f878b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:35.816887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 222/427] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:35 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:35.819096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b582c-1993-49be-80bc-0e21d69e84db None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:35.819692 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b582c-1993-49be-80bc-0e21d69e84db None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:35.820298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b582c-1993-49be-80bc-0e21d69e84db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:35.850426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e12b582c-1993-49be-80bc-0e21d69e84db None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:35.851989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e12b582c-1993-49be-80bc-0e21d69e84db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:35.900498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:35.930729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 206/428] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:35.931540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72e89e-6a4d-49bb-9fea-9ebc79584204 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:35.958446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 223/429] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:35.959464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b582c-1993-49be-80bc-0e21d69e84db None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:35.993144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:36.040615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72e89e-6a4d-49bb-9fea-9ebc79584204 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.074891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b582c-1993-49be-80bc-0e21d69e84db None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.079037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d26728-ab71-48f0-86f2-9f555a59ca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.079721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d26728-ab71-48f0-86f2-9f555a59ca9b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.080061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d26728-ab71-48f0-86f2-9f555a59ca9b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.080445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d26728-ab71-48f0-86f2-9f555a59ca9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.081278 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d26728-ab71-48f0-86f2-9f555a59ca9b None admin] RBAC: Adding query filter params (group.id=e8fb9a57bff44af2a0632fc095e16248, role.id=13913230583d4ce384599ac5dace7ffa, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:36.081706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d26728-ab71-48f0-86f2-9f555a59ca9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.083268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d26728-ab71-48f0-86f2-9f555a59ca9b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.090139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 207/430] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Aug 24 09:57:35 2022] GET /identity/v3/role_assignments?group.id=e8fb9a57bff44af2a0632fc095e16248&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=13913230583d4ce384599ac5dace7ffa => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:36.120388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37c168-fb90-4375-8e5e-89b52b0b7b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.121069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37c168-fb90-4375-8e5e-89b52b0b7b2d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.121353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37c168-fb90-4375-8e5e-89b52b0b7b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.121628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37c168-fb90-4375-8e5e-89b52b0b7b2d None admin] PATH_INFO: `/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.124621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37c168-fb90-4375-8e5e-89b52b0b7b2d None admin] RBAC: Authorizing `identity:get_project(project_id=c4bdbfd07cd94eeea5087a9d39f83369)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.126319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37c168-fb90-4375-8e5e-89b52b0b7b2d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.128333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 224/431] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:35 2022] GET /identity/v3/projects/c4bdbfd07cd94eeea5087a9d39f83369 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:36.134611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d26728-ab71-48f0-86f2-9f555a59ca9b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.295768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d26728-ab71-48f0-86f2-9f555a59ca9b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.319742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:36.341817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e19ab-60af-4751-a441-ac9c07b7bd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.342448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e19ab-60af-4751-a441-ac9c07b7bd72 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.342724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e19ab-60af-4751-a441-ac9c07b7bd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.342996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e19ab-60af-4751-a441-ac9c07b7bd72 None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.344185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e19ab-60af-4751-a441-ac9c07b7bd72 None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.345686 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e19ab-60af-4751-a441-ac9c07b7bd72 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.347735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 208/432] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:36 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:36.356542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37c168-fb90-4375-8e5e-89b52b0b7b2d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.395625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:36.408279 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e19ab-60af-4751-a441-ac9c07b7bd72 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.491313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37c168-fb90-4375-8e5e-89b52b0b7b2d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.529335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9187a-7b22-469d-b251-88f6d46bb730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.530005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9187a-7b22-469d-b251-88f6d46bb730 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.530287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9187a-7b22-469d-b251-88f6d46bb730 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.530655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9187a-7b22-469d-b251-88f6d46bb730 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.531407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9187a-7b22-469d-b251-88f6d46bb730 None admin] RBAC: Adding query filter params (group.id=e8fb9a57bff44af2a0632fc095e16248, role.id=276ae07995d14b5d8ceb8e646392c2b1, scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:36.531712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9187a-7b22-469d-b251-88f6d46bb730 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.533257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9187a-7b22-469d-b251-88f6d46bb730 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.545272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e19ab-60af-4751-a441-ac9c07b7bd72 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.548438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 225/433] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Aug 24 09:57:36 2022] GET /identity/v3/role_assignments?group.id=e8fb9a57bff44af2a0632fc095e16248&scope.project.id=c4bdbfd07cd94eeea5087a9d39f83369&role.id=276ae07995d14b5d8ceb8e646392c2b1 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:36.594312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0e915-a6b0-4d0d-835d-16d74c70fb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.595034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0e915-a6b0-4d0d-835d-16d74c70fb12 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.595310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0e915-a6b0-4d0d-835d-16d74c70fb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.595584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0e915-a6b0-4d0d-835d-16d74c70fb12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.609118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0e915-a6b0-4d0d-835d-16d74c70fb12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-4cc0e915-a6b0-4d0d-835d-16d74c70fb12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:36.615588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:36.616684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:36.618169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 209/434] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:36.622847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d15a4-37ab-4259-8669-1f8e0ea61a6e None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.623403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0e915-a6b0-4d0d-835d-16d74c70fb12 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.623553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d15a4-37ab-4259-8669-1f8e0ea61a6e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.623553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d15a4-37ab-4259-8669-1f8e0ea61a6e None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.629584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 226/435] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:36.632141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b3536-c001-4cf3-80c2-13b52ebef76b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.632676 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b3536-c001-4cf3-80c2-13b52ebef76b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.633066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b3536-c001-4cf3-80c2-13b52ebef76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.655201 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6f0b3536-c001-4cf3-80c2-13b52ebef76b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:36.656498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6f0b3536-c001-4cf3-80c2-13b52ebef76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:36.689155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:36.746226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 227/436] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:36.766937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0e915-a6b0-4d0d-835d-16d74c70fb12 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.809182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc365f49-52e5-480c-8966-44cdf2e495de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.809818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc365f49-52e5-480c-8966-44cdf2e495de None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.810093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc365f49-52e5-480c-8966-44cdf2e495de None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.810365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc365f49-52e5-480c-8966-44cdf2e495de None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.811056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc365f49-52e5-480c-8966-44cdf2e495de None admin] RBAC: Adding query filter params (name=member) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:36.811342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc365f49-52e5-480c-8966-44cdf2e495de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.812710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc365f49-52e5-480c-8966-44cdf2e495de None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.821920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 210/437] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:57:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:36.827655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b3536-c001-4cf3-80c2-13b52ebef76b None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.852333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1dd56-18bd-4ef4-8b15-d2e1b1353ed8 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.852846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1dd56-18bd-4ef4-8b15-d2e1b1353ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.853279 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1dd56-18bd-4ef4-8b15-d2e1b1353ed8 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.854774 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 211/438] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:57:36.861022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a4207-935f-4b62-af5f-131789216228 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.861514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a4207-935f-4b62-af5f-131789216228 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.861897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a4207-935f-4b62-af5f-131789216228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.863502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:36.886446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3a0a4207-935f-4b62-af5f-131789216228 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:36.891649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3a0a4207-935f-4b62-af5f-131789216228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:36.975737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b3536-c001-4cf3-80c2-13b52ebef76b None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.995852 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 212/439] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:57:37.027738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee515fd-abf6-4d2f-9641-bc488241b205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.028344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee515fd-abf6-4d2f-9641-bc488241b205 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.028621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee515fd-abf6-4d2f-9641-bc488241b205 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.028910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee515fd-abf6-4d2f-9641-bc488241b205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.048437 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a4207-935f-4b62-af5f-131789216228 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.103533 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:37.141635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee515fd-abf6-4d2f-9641-bc488241b205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.143128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee515fd-abf6-4d2f-9641-bc488241b205 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.188369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:37.206796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a4207-935f-4b62-af5f-131789216228 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.249884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02837e2-f258-4ae2-9600-15f2c4551072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.250517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02837e2-f258-4ae2-9600-15f2c4551072 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.250795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02837e2-f258-4ae2-9600-15f2c4551072 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.251113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02837e2-f258-4ae2-9600-15f2c4551072 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.262336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02837e2-f258-4ae2-9600-15f2c4551072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-d02837e2-f258-4ae2-9600-15f2c4551072 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:37.267125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:37.268011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:37.268794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 213/440] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:37.274936 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02837e2-f258-4ae2-9600-15f2c4551072 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.285757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 228/441] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:37.286625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee515fd-abf6-4d2f-9641-bc488241b205 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.383436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:37.386846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02837e2-f258-4ae2-9600-15f2c4551072 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.394138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee515fd-abf6-4d2f-9641-bc488241b205 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.425918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e969e-31c0-4f3a-9f37-55b244be81f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.426163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:37.426515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e969e-31c0-4f3a-9f37-55b244be81f6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.426788 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e969e-31c0-4f3a-9f37-55b244be81f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.427054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e969e-31c0-4f3a-9f37-55b244be81f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.427972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e969e-31c0-4f3a-9f37-55b244be81f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:37.428195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e969e-31c0-4f3a-9f37-55b244be81f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.429811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e969e-31c0-4f3a-9f37-55b244be81f6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.434992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29b7f5-60a7-4bae-99f9-100f4100fc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.439328 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29b7f5-60a7-4bae-99f9-100f4100fc16 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.439328 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29b7f5-60a7-4bae-99f9-100f4100fc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.439328 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29b7f5-60a7-4bae-99f9-100f4100fc16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.440009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 214/442] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:37.440009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e969e-31c0-4f3a-9f37-55b244be81f6 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.448666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29b7f5-60a7-4bae-99f9-100f4100fc16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-ee29b7f5-60a7-4bae-99f9-100f4100fc16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:37.453628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:37.454865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 229/443] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:37.456196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29b7f5-60a7-4bae-99f9-100f4100fc16 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.554036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e969e-31c0-4f3a-9f37-55b244be81f6 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.566698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29b7f5-60a7-4bae-99f9-100f4100fc16 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.588086 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178ee9e-c7cb-44af-8534-0152f42d3305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.588909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178ee9e-c7cb-44af-8534-0152f42d3305 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.589188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178ee9e-c7cb-44af-8534-0152f42d3305 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.589453 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178ee9e-c7cb-44af-8534-0152f42d3305 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.591576 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178ee9e-c7cb-44af-8534-0152f42d3305 None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.592035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178ee9e-c7cb-44af-8534-0152f42d3305 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.594380 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 215/444] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:37.594511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178ee9e-c7cb-44af-8534-0152f42d3305 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.610406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25779b8-f128-4eb6-bee8-cf3274b808b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.611015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25779b8-f128-4eb6-bee8-cf3274b808b9 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.611290 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25779b8-f128-4eb6-bee8-cf3274b808b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.611562 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25779b8-f128-4eb6-bee8-cf3274b808b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.624723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:37.678207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:37.690223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25779b8-f128-4eb6-bee8-cf3274b808b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.691712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25779b8-f128-4eb6-bee8-cf3274b808b9 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.704833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178ee9e-c7cb-44af-8534-0152f42d3305 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.744669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c211fb-3d40-4e7e-aecb-8a48b2363f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.745303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c211fb-3d40-4e7e-aecb-8a48b2363f58 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.745586 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c211fb-3d40-4e7e-aecb-8a48b2363f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.745874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c211fb-3d40-4e7e-aecb-8a48b2363f58 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.746534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c211fb-3d40-4e7e-aecb-8a48b2363f58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:37.746925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c211fb-3d40-4e7e-aecb-8a48b2363f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.748353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c211fb-3d40-4e7e-aecb-8a48b2363f58 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.758089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 216/445] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:37.758939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c211fb-3d40-4e7e-aecb-8a48b2363f58 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.801079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 230/446] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:37.802243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25779b8-f128-4eb6-bee8-cf3274b808b9 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.865117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:37.875893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c211fb-3d40-4e7e-aecb-8a48b2363f58 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.898754 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:37.910784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25779b8-f128-4eb6-bee8-cf3274b808b9 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.913561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab65eb-8f4e-42fb-9795-f17d112655ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.914536 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab65eb-8f4e-42fb-9795-f17d112655ad None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.914986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab65eb-8f4e-42fb-9795-f17d112655ad None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.915654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab65eb-8f4e-42fb-9795-f17d112655ad None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.916808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab65eb-8f4e-42fb-9795-f17d112655ad None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.918282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab65eb-8f4e-42fb-9795-f17d112655ad None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.919975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 217/447] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:37.925457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab65eb-8f4e-42fb-9795-f17d112655ad None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.950052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149088ee-197c-4f43-93a7-7978c7187389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.950584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149088ee-197c-4f43-93a7-7978c7187389 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.950862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149088ee-197c-4f43-93a7-7978c7187389 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.951134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149088ee-197c-4f43-93a7-7978c7187389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:38.030021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab65eb-8f4e-42fb-9795-f17d112655ad None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:38.039573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149088ee-197c-4f43-93a7-7978c7187389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:38.041070 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149088ee-197c-4f43-93a7-7978c7187389 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:38.065900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f0eb0-92ab-4fd4-8430-22f60cbc0426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:38.066528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f0eb0-92ab-4fd4-8430-22f60cbc0426 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:38.066801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f0eb0-92ab-4fd4-8430-22f60cbc0426 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:38.067071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f0eb0-92ab-4fd4-8430-22f60cbc0426 None admin] PATH_INFO: `/v3/projects/85765b8a176a42c8b2b1e092bbcf9c4f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:38.068805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f0eb0-92ab-4fd4-8430-22f60cbc0426 None admin] RBAC: Authorizing `identity:get_project(project_id=85765b8a176a42c8b2b1e092bbcf9c4f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:38.070393 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f0eb0-92ab-4fd4-8430-22f60cbc0426 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:38.072042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 218/448] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/projects/85765b8a176a42c8b2b1e092bbcf9c4f => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:38.072795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f0eb0-92ab-4fd4-8430-22f60cbc0426 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:38.101309 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:38.140555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:38.156163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 231/449] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:38.157048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149088ee-197c-4f43-93a7-7978c7187389 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:38.181239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f0eb0-92ab-4fd4-8430-22f60cbc0426 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:38.216420 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb317b37-4b2c-4f08-b0ef-2b894c6ecc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:38.216961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb317b37-4b2c-4f08-b0ef-2b894c6ecc12 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:38.217239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb317b37-4b2c-4f08-b0ef-2b894c6ecc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:38.217510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb317b37-4b2c-4f08-b0ef-2b894c6ecc12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:38.218288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb317b37-4b2c-4f08-b0ef-2b894c6ecc12 None admin] RBAC: Adding query filter params (group.id=e8fb9a57bff44af2a0632fc095e16248, role.id=13913230583d4ce384599ac5dace7ffa, scope.project.id=42ce709191ce4b1b961e89e218566b5b) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:38.218591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb317b37-4b2c-4f08-b0ef-2b894c6ecc12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:38.220024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb317b37-4b2c-4f08-b0ef-2b894c6ecc12 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:38.226728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 219/450] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Aug 24 09:57:38 2022] GET /identity/v3/role_assignments?group.id=e8fb9a57bff44af2a0632fc095e16248&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=13913230583d4ce384599ac5dace7ffa => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:38.227547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb317b37-4b2c-4f08-b0ef-2b894c6ecc12 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:38.275586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149088ee-197c-4f43-93a7-7978c7187389 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:38.319670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb741b-c5f1-4a81-8a18-b32f66c06c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:38.320364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb741b-c5f1-4a81-8a18-b32f66c06c65 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:38.320644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb741b-c5f1-4a81-8a18-b32f66c06c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:38.320972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb741b-c5f1-4a81-8a18-b32f66c06c65 None admin] PATH_INFO: `/v3/groups/06f83eb2b41b4ccabcc238999d04b881` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:38.323723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb741b-c5f1-4a81-8a18-b32f66c06c65 None admin] RBAC: Authorizing `identity:get_group(group_id=06f83eb2b41b4ccabcc238999d04b881)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:38.325300 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb741b-c5f1-4a81-8a18-b32f66c06c65 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:38.329943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 232/451] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:38 2022] GET /identity/v3/groups/06f83eb2b41b4ccabcc238999d04b881 => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:38.333319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb741b-c5f1-4a81-8a18-b32f66c06c65 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:38.347157 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:38.359042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb317b37-4b2c-4f08-b0ef-2b894c6ecc12 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:38.401971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:38.404939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388c436-be8c-4ea6-9f87-217f00abe393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:38.405590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388c436-be8c-4ea6-9f87-217f00abe393 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:38.405863 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388c436-be8c-4ea6-9f87-217f00abe393 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:38.406138 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388c436-be8c-4ea6-9f87-217f00abe393 None admin] PATH_INFO: `/v3/projects/42ce709191ce4b1b961e89e218566b5b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:38.407281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388c436-be8c-4ea6-9f87-217f00abe393 None admin] RBAC: Authorizing `identity:get_project(project_id=42ce709191ce4b1b961e89e218566b5b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:38.408737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388c436-be8c-4ea6-9f87-217f00abe393 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:38.411173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 220/452] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:57:38 2022] GET /identity/v3/projects/42ce709191ce4b1b961e89e218566b5b => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:38.417031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388c436-be8c-4ea6-9f87-217f00abe393 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:38.455637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb741b-c5f1-4a81-8a18-b32f66c06c65 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:38.500045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f2aac-1375-4916-a2ce-a9156352e5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:38.500804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f2aac-1375-4916-a2ce-a9156352e5dc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:38.500952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f2aac-1375-4916-a2ce-a9156352e5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:38.501154 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f2aac-1375-4916-a2ce-a9156352e5dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:38.501921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f2aac-1375-4916-a2ce-a9156352e5dc None admin] RBAC: Adding query filter params (group.id=06f83eb2b41b4ccabcc238999d04b881, role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=85765b8a176a42c8b2b1e092bbcf9c4f) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:38.503121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f2aac-1375-4916-a2ce-a9156352e5dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:38.504574 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f2aac-1375-4916-a2ce-a9156352e5dc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:38.513204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 233/453] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Aug 24 09:57:38 2022] GET /identity/v3/role_assignments?group.id=06f83eb2b41b4ccabcc238999d04b881&scope.project.id=85765b8a176a42c8b2b1e092bbcf9c4f&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:38.561320 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388c436-be8c-4ea6-9f87-217f00abe393 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:38.606888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c643590-738d-408e-944a-f99c0cc282f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:38.607549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643590-738d-408e-944a-f99c0cc282f9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:38.607910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643590-738d-408e-944a-f99c0cc282f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:38.608115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643590-738d-408e-944a-f99c0cc282f9 None admin] PATH_INFO: `/v3/groups/e8fb9a57bff44af2a0632fc095e16248` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:38.609429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c643590-738d-408e-944a-f99c0cc282f9 None admin] RBAC: Authorizing `identity:get_group(group_id=e8fb9a57bff44af2a0632fc095e16248)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:38.611130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c643590-738d-408e-944a-f99c0cc282f9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:38.612794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 221/454] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Aug 24 09:57:38 2022] GET /identity/v3/groups/e8fb9a57bff44af2a0632fc095e16248 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:38.619552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f2aac-1375-4916-a2ce-a9156352e5dc None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:38.739254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f2aac-1375-4916-a2ce-a9156352e5dc None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:38.762761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:38.776918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae671ced-4554-4f54-aaaf-496231416cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:38.777847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae671ced-4554-4f54-aaaf-496231416cac None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:38.777847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae671ced-4554-4f54-aaaf-496231416cac None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:38.778099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae671ced-4554-4f54-aaaf-496231416cac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:38.778829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae671ced-4554-4f54-aaaf-496231416cac None admin] RBAC: Adding query filter params (group.id=e8fb9a57bff44af2a0632fc095e16248, role.id=276ae07995d14b5d8ceb8e646392c2b1, scope.project.id=42ce709191ce4b1b961e89e218566b5b) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:38.779135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae671ced-4554-4f54-aaaf-496231416cac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:38.780553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae671ced-4554-4f54-aaaf-496231416cac None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:38.786286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 234/455] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Aug 24 09:57:38 2022] GET /identity/v3/role_assignments?group.id=e8fb9a57bff44af2a0632fc095e16248&scope.project.id=42ce709191ce4b1b961e89e218566b5b&role.id=276ae07995d14b5d8ceb8e646392c2b1 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:41.409255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9911a20-ed29-45d9-86a0-797a8b48c548 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:41.409812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9911a20-ed29-45d9-86a0-797a8b48c548 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:41.409812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9911a20-ed29-45d9-86a0-797a8b48c548 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:41.411297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 222/456] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:41.417950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf27400-b349-42a9-9fb3-1dd1cb5ba187 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:41.418411 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf27400-b349-42a9-9fb3-1dd1cb5ba187 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:41.418512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf27400-b349-42a9-9fb3-1dd1cb5ba187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:41.444750 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-edf27400-b349-42a9-9fb3-1dd1cb5ba187 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:41.450171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-edf27400-b349-42a9-9fb3-1dd1cb5ba187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:41.486779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f9edd-940a-4a35-861e-740d9a141e74 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:41.487383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f9edd-940a-4a35-861e-740d9a141e74 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:41.490278 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f9edd-940a-4a35-861e-740d9a141e74 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:41.490278 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 223/457] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:57:41.495827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6f576-51c0-4955-ab4a-dfffdf9e1cd2 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:41.496112 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6f576-51c0-4955-ab4a-dfffdf9e1cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:41.496390 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6f576-51c0-4955-ab4a-dfffdf9e1cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:41.516526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3ac6f576-51c0-4955-ab4a-dfffdf9e1cd2 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:41.518025 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3ac6f576-51c0-4955-ab4a-dfffdf9e1cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:41.541599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:41.567582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 235/458] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:41.607477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 224/459] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:41.911996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf27400-b349-42a9-9fb3-1dd1cb5ba187 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:41.971311 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52cc9d-c2ad-458d-bae5-e3b9acf5320c None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:41.971932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52cc9d-c2ad-458d-bae5-e3b9acf5320c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:41.972452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52cc9d-c2ad-458d-bae5-e3b9acf5320c None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:41.974521 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 225/460] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:41.980403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665338fc-3893-4d54-96dc-352473acbb96 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:41.981102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665338fc-3893-4d54-96dc-352473acbb96 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:41.981713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665338fc-3893-4d54-96dc-352473acbb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:42.025234 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-665338fc-3893-4d54-96dc-352473acbb96 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:42.026840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-665338fc-3893-4d54-96dc-352473acbb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:42.042292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf27400-b349-42a9-9fb3-1dd1cb5ba187 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:42.058617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:42.080301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d44ca5-3c6c-4de3-9c71-7daa99dfb5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:42.080963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d44ca5-3c6c-4de3-9c71-7daa99dfb5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:42.081412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d44ca5-3c6c-4de3-9c71-7daa99dfb5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:42.081833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d44ca5-3c6c-4de3-9c71-7daa99dfb5a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:42.093294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d44ca5-3c6c-4de3-9c71-7daa99dfb5a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:42.094743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d44ca5-3c6c-4de3-9c71-7daa99dfb5a7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:42.109662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 226/461] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:42.111005 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665338fc-3893-4d54-96dc-352473acbb96 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-37d44ca5-3c6c-4de3-9c71-7daa99dfb5a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:42.116552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:42.118492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:42.120075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:42.120075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:42.120075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:42.120075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 236/462] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:42.128899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d44ca5-3c6c-4de3-9c71-7daa99dfb5a7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:42.222538 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:42.235316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665338fc-3893-4d54-96dc-352473acbb96 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:42.274922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117dd2d-3bb8-4873-be4d-4643d6fadf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:42.275894 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117dd2d-3bb8-4873-be4d-4643d6fadf06 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:42.276375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117dd2d-3bb8-4873-be4d-4643d6fadf06 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:42.276883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117dd2d-3bb8-4873-be4d-4643d6fadf06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:42.280548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d44ca5-3c6c-4de3-9c71-7daa99dfb5a7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:42.288739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117dd2d-3bb8-4873-be4d-4643d6fadf06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:42.290601 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117dd2d-3bb8-4873-be4d-4643d6fadf06 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-7117dd2d-3bb8-4873-be4d-4643d6fadf06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:42.307957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:42.309008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:42.309855 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:42.309855 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:42.309855 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:42.311957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 227/463] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:42.313532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3fce0-aa4f-4904-a886-63f307bf3f82 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:42.314037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3fce0-aa4f-4904-a886-63f307bf3f82 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:42.314520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3fce0-aa4f-4904-a886-63f307bf3f82 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:42.316932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 228/464] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:42.319554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3fce0-aa4f-4904-a886-63f307bf3f82 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:42.323135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb6cca-058e-405f-a17c-654ef471a74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:42.323766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb6cca-058e-405f-a17c-654ef471a74f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:42.324071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb6cca-058e-405f-a17c-654ef471a74f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:42.329644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb6cca-058e-405f-a17c-654ef471a74f None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:42.329644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb6cca-058e-405f-a17c-654ef471a74f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:42.329644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb6cca-058e-405f-a17c-654ef471a74f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:42.329644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb6cca-058e-405f-a17c-654ef471a74f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:42.345930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 237/465] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:42.347311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c093a-2f27-4785-a04a-1b83abda03c2 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:42.347604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c093a-2f27-4785-a04a-1b83abda03c2 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:42.347883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c093a-2f27-4785-a04a-1b83abda03c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:42.368707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e28c093a-2f27-4785-a04a-1b83abda03c2 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:42.369954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e28c093a-2f27-4785-a04a-1b83abda03c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:42.393642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:42.445675 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3fce0-aa4f-4904-a886-63f307bf3f82 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:42.448379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 238/466] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:42.449260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c093a-2f27-4785-a04a-1b83abda03c2 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:42.495060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629b2c0-02ef-4b8f-99e9-9fe0c301d5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:42.496062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629b2c0-02ef-4b8f-99e9-9fe0c301d5da None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:42.496537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629b2c0-02ef-4b8f-99e9-9fe0c301d5da None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:42.497123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629b2c0-02ef-4b8f-99e9-9fe0c301d5da None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:42.498016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629b2c0-02ef-4b8f-99e9-9fe0c301d5da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:42.498510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629b2c0-02ef-4b8f-99e9-9fe0c301d5da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:42.500326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629b2c0-02ef-4b8f-99e9-9fe0c301d5da None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:42.508739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:42.520044 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 229/467] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:42.522060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629b2c0-02ef-4b8f-99e9-9fe0c301d5da None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:42.558618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c093a-2f27-4785-a04a-1b83abda03c2 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:42.595147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133229f-b397-4c89-8ab3-6cc0280f4aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:42.595758 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133229f-b397-4c89-8ab3-6cc0280f4aa9 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:42.596073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133229f-b397-4c89-8ab3-6cc0280f4aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:42.596309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133229f-b397-4c89-8ab3-6cc0280f4aa9 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:42.596983 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133229f-b397-4c89-8ab3-6cc0280f4aa9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:42.598448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133229f-b397-4c89-8ab3-6cc0280f4aa9 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:42.638019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629b2c0-02ef-4b8f-99e9-9fe0c301d5da None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:42.678040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7250ed-f511-4077-87e6-0ad59e55907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:42.678691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7250ed-f511-4077-87e6-0ad59e55907e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:42.678961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7250ed-f511-4077-87e6-0ad59e55907e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:42.679224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7250ed-f511-4077-87e6-0ad59e55907e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:42.690031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7250ed-f511-4077-87e6-0ad59e55907e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:42.691473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7250ed-f511-4077-87e6-0ad59e55907e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-2f7250ed-f511-4077-87e6-0ad59e55907e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:42.707166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:42.708132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:42.713968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:42.713968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:42.713968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:42.713968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 230/468] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:42.713968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7250ed-f511-4077-87e6-0ad59e55907e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:42.772971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 239/469] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:57:42 2022] POST /identity/v3/users => generated 246 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:42.772971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133229f-b397-4c89-8ab3-6cc0280f4aa9 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:42.777132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:42.790488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:42.848484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7250ed-f511-4077-87e6-0ad59e55907e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:42.892425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854438ce-8977-4cf3-9998-fd54a45ae1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:42.893228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854438ce-8977-4cf3-9998-fd54a45ae1c0 None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:42.893409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854438ce-8977-4cf3-9998-fd54a45ae1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:42.893798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854438ce-8977-4cf3-9998-fd54a45ae1c0 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:42.894508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854438ce-8977-4cf3-9998-fd54a45ae1c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:42.896426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854438ce-8977-4cf3-9998-fd54a45ae1c0 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:42.919178 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133229f-b397-4c89-8ab3-6cc0280f4aa9 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:42.960002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c6df0-759b-4503-949d-52425e320642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:42.961761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c6df0-759b-4503-949d-52425e320642 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:42.961761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c6df0-759b-4503-949d-52425e320642 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:42.961761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c6df0-759b-4503-949d-52425e320642 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:42.962428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c6df0-759b-4503-949d-52425e320642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:42.962805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c6df0-759b-4503-949d-52425e320642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:42.964248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c6df0-759b-4503-949d-52425e320642 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:42.978791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 240/470] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:42.979725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c6df0-759b-4503-949d-52425e320642 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:43.025135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 231/471] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:57:42 2022] POST /identity/v3/users => generated 248 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:43.025345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e970d6-3af7-4a37-b3fc-e9463199d8fc None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:43.025485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e970d6-3af7-4a37-b3fc-e9463199d8fc None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:43.025814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e970d6-3af7-4a37-b3fc-e9463199d8fc None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:43.027338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 232/472] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:43.032883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e970d6-3af7-4a37-b3fc-e9463199d8fc None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:43.069452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:43.109650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c6df0-759b-4503-949d-52425e320642 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:43.165433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be736aca-0c2b-40b8-b9ba-46f3de62cac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:43.166436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be736aca-0c2b-40b8-b9ba-46f3de62cac0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:43.166898 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be736aca-0c2b-40b8-b9ba-46f3de62cac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:43.167397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be736aca-0c2b-40b8-b9ba-46f3de62cac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:43.181691 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be736aca-0c2b-40b8-b9ba-46f3de62cac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:43.183669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be736aca-0c2b-40b8-b9ba-46f3de62cac0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:43.192525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e970d6-3af7-4a37-b3fc-e9463199d8fc None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:43.202035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-be736aca-0c2b-40b8-b9ba-46f3de62cac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:43.203812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:43.204713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:43.205613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:43.205613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:43.205613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:43.211277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 241/473] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:43.212773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f40204-1789-4002-a975-d353fe13a24b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:43.213079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f40204-1789-4002-a975-d353fe13a24b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:43.213358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f40204-1789-4002-a975-d353fe13a24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:43.233363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-f5f40204-1789-4002-a975-d353fe13a24b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:43.234693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f5f40204-1789-4002-a975-d353fe13a24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:43.241174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e3647-a131-4712-b724-27d5ae7b0604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:43.242123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e3647-a131-4712-b724-27d5ae7b0604 None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:43.242456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e3647-a131-4712-b724-27d5ae7b0604 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:43.242727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e3647-a131-4712-b724-27d5ae7b0604 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:43.243399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e3647-a131-4712-b724-27d5ae7b0604 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:43.245598 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e3647-a131-4712-b724-27d5ae7b0604 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:43.321558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 242/474] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:43.322857 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f40204-1789-4002-a975-d353fe13a24b None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:43.388521 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 233/475] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:57:43 2022] POST /identity/v3/users => generated 248 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:43.398029 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:43.457334 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f40204-1789-4002-a975-d353fe13a24b None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:43.515235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9a245-9506-4eee-a132-cfb77d35ed61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:43.515922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9a245-9506-4eee-a132-cfb77d35ed61 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:43.516198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9a245-9506-4eee-a132-cfb77d35ed61 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:43.516469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9a245-9506-4eee-a132-cfb77d35ed61 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:43.517147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9a245-9506-4eee-a132-cfb77d35ed61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:43.517448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9a245-9506-4eee-a132-cfb77d35ed61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:43.518818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9a245-9506-4eee-a132-cfb77d35ed61 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:43.536182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 243/476] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:43.543203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e3647-a131-4712-b724-27d5ae7b0604 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:43.673545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e3647-a131-4712-b724-27d5ae7b0604 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:43.678506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9a245-9506-4eee-a132-cfb77d35ed61 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:43.721865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43475c3-ede7-43c6-968d-9497f13f1dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:43.722319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43475c3-ede7-43c6-968d-9497f13f1dc8 None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:43.722603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43475c3-ede7-43c6-968d-9497f13f1dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:43.722876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43475c3-ede7-43c6-968d-9497f13f1dc8 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:43.723581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43475c3-ede7-43c6-968d-9497f13f1dc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:43.725037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43475c3-ede7-43c6-968d-9497f13f1dc8 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:43.814756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9a245-9506-4eee-a132-cfb77d35ed61 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:43.828621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 234/477] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:57:43 2022] POST /identity/v3/users => generated 249 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:57:43.839798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:43.864539 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4f2bc-06ef-4e48-bb79-76bfb9a83465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:43.866107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4f2bc-06ef-4e48-bb79-76bfb9a83465 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:43.866587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4f2bc-06ef-4e48-bb79-76bfb9a83465 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:43.867055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4f2bc-06ef-4e48-bb79-76bfb9a83465 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:43.868068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4f2bc-06ef-4e48-bb79-76bfb9a83465 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:43.869774 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4f2bc-06ef-4e48-bb79-76bfb9a83465 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:43.919807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.sql.core [None req-1eb4f2bc-06ef-4e48-bb79-76bfb9a83465 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:57:43.919807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 09:57:43.919807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [parameters: {'id': '88da9d95998b41498406927364a87cd0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 09:57:43.919807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81931) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-1eb4f2bc-06ef-4e48-bb79-76bfb9a83465 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application result.read() Aug 24 09:57:43.941023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:57:43.942458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application yield resource Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise self.value Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application session.commit() Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:57:43.944263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:57:43.945743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:57:43.947155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application result.read() Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:57:43.948260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [parameters: {'id': '88da9d95998b41498406927364a87cd0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:43.955121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 09:57:43.957123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:43.959009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 244/478] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Aug 24 09:57:43 2022] POST /identity/v3/roles => generated 144 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:43.959112 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43475c3-ede7-43c6-968d-9497f13f1dc8 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:43.980250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:44.092039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43475c3-ede7-43c6-968d-9497f13f1dc8 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:44.129193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb20ec-605c-47f5-ac54-be6c2e778676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:44.129871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb20ec-605c-47f5-ac54-be6c2e778676 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:44.130154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb20ec-605c-47f5-ac54-be6c2e778676 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:44.130457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb20ec-605c-47f5-ac54-be6c2e778676 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:44.141621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb20ec-605c-47f5-ac54-be6c2e778676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-89eb20ec-605c-47f5-ac54-be6c2e778676 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:44.148614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:44.149545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:44.150354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 235/479] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:57:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:44.156732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4f2bc-06ef-4e48-bb79-76bfb9a83465 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:44.288572 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4f2bc-06ef-4e48-bb79-76bfb9a83465 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:44.341668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9551a-ff66-4f18-a771-77bc61e6e3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:44.342369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9551a-ff66-4f18-a771-77bc61e6e3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:44.342664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9551a-ff66-4f18-a771-77bc61e6e3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:44.342949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9551a-ff66-4f18-a771-77bc61e6e3f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:44.343636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9551a-ff66-4f18-a771-77bc61e6e3f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:44.343947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9551a-ff66-4f18-a771-77bc61e6e3f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:44.345368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9551a-ff66-4f18-a771-77bc61e6e3f4 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:44.355295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 245/480] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:45.082765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31612b-158b-4a8e-834a-a8e064e319fe None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:45.083834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31612b-158b-4a8e-834a-a8e064e319fe None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:45.084373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31612b-158b-4a8e-834a-a8e064e319fe None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:45.086622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 236/481] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:45.092577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba88d-4b88-4460-b97f-72a1f4ab788b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:45.093217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba88d-4b88-4460-b97f-72a1f4ab788b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:45.093389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba88d-4b88-4460-b97f-72a1f4ab788b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:45.125497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6abba88d-4b88-4460-b97f-72a1f4ab788b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:45.127140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6abba88d-4b88-4460-b97f-72a1f4ab788b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:45.184973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:45.234222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 246/482] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:45.487274 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f433425-84f3-4b1e-b6de-e2c4365ca20b None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:45.487640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f433425-84f3-4b1e-b6de-e2c4365ca20b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:45.487898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f433425-84f3-4b1e-b6de-e2c4365ca20b None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:45.489604 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 237/483] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:45.496372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05561da-cf01-49a0-9a6a-5660cf298679 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:45.496669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05561da-cf01-49a0-9a6a-5660cf298679 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:45.496957 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05561da-cf01-49a0-9a6a-5660cf298679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:45.522359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c05561da-cf01-49a0-9a6a-5660cf298679 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:45.523542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c05561da-cf01-49a0-9a6a-5660cf298679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:45.622913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 247/484] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:45.718847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f433425-84f3-4b1e-b6de-e2c4365ca20b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:45.844930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:45.874921 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f433425-84f3-4b1e-b6de-e2c4365ca20b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:45.927071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4de2a-9923-45e4-8caf-42056f936f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:45.927783 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4de2a-9923-45e4-8caf-42056f936f06 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:45.928212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4de2a-9923-45e4-8caf-42056f936f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:45.928489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4de2a-9923-45e4-8caf-42056f936f06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:45.947854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4de2a-9923-45e4-8caf-42056f936f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b9b4de2a-9923-45e4-8caf-42056f936f06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:45.952609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:45.953549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:45.954757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:45.957573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 238/485] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:45.961136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05561da-cf01-49a0-9a6a-5660cf298679 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:46.033735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4de2a-9923-45e4-8caf-42056f936f06 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:46.089660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05561da-cf01-49a0-9a6a-5660cf298679 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:46.092828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:46.144267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6a7e8-989a-4906-8b62-b8f57f29e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:46.144904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6a7e8-989a-4906-8b62-b8f57f29e08b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:46.145187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6a7e8-989a-4906-8b62-b8f57f29e08b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:46.145458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6a7e8-989a-4906-8b62-b8f57f29e08b None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:46.148900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6a7e8-989a-4906-8b62-b8f57f29e08b None admin] RBAC: Adding query filter params (name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:46.148900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6a7e8-989a-4906-8b62-b8f57f29e08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:46.148900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6a7e8-989a-4906-8b62-b8f57f29e08b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:46.162719 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 248/486] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:57:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:46.164228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6a7e8-989a-4906-8b62-b8f57f29e08b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:46.242623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4de2a-9923-45e4-8caf-42056f936f06 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:46.287007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8c6c8-7ed4-4757-88cc-7b960c503803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:46.287650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8c6c8-7ed4-4757-88cc-7b960c503803 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:46.288188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8c6c8-7ed4-4757-88cc-7b960c503803 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:46.288463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8c6c8-7ed4-4757-88cc-7b960c503803 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:46.303919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:46.311068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8c6c8-7ed4-4757-88cc-7b960c503803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b1d8c6c8-7ed4-4757-88cc-7b960c503803 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:46.325155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:46.326118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:46.326907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:46.327527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 239/487] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:46.334286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8c6c8-7ed4-4757-88cc-7b960c503803 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:46.375259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6a7e8-989a-4906-8b62-b8f57f29e08b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:46.451292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006eb35-231d-4876-9ecc-dc42a7ceaab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:46.452280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006eb35-231d-4876-9ecc-dc42a7ceaab7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:46.452783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006eb35-231d-4876-9ecc-dc42a7ceaab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:46.453288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006eb35-231d-4876-9ecc-dc42a7ceaab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:46.465436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006eb35-231d-4876-9ecc-dc42a7ceaab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:46.467263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006eb35-231d-4876-9ecc-dc42a7ceaab7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-5006eb35-231d-4876-9ecc-dc42a7ceaab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:46.483882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:46.485188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:46.486109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:46.486109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:46.486109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:46.488403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 249/488] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:46.490590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3999e25-b3ff-45ed-9b7a-93cd01e92bac None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:46.491107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3999e25-b3ff-45ed-9b7a-93cd01e92bac None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:46.491617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3999e25-b3ff-45ed-9b7a-93cd01e92bac None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:46.493160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 250/489] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:46.494418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997f70c-71c4-477e-a60b-897e313453bc None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:46.495008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997f70c-71c4-477e-a60b-897e313453bc None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:46.495481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997f70c-71c4-477e-a60b-897e313453bc None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:46.497370 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 251/490] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:46.498347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997f70c-71c4-477e-a60b-897e313453bc None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:46.515427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8c6c8-7ed4-4757-88cc-7b960c503803 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:46.531115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:46.567014 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdae547-9983-4e88-a872-92280e451fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:46.567778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdae547-9983-4e88-a872-92280e451fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:46.568603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdae547-9983-4e88-a872-92280e451fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:46.570301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdae547-9983-4e88-a872-92280e451fc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:46.570301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdae547-9983-4e88-a872-92280e451fc6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:46.570301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdae547-9983-4e88-a872-92280e451fc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:46.571671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdae547-9983-4e88-a872-92280e451fc6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:46.580897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 240/491] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:57:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:46.587365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676649ff-bcae-4186-8bfc-e9ff7efd6419 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:46.587641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676649ff-bcae-4186-8bfc-e9ff7efd6419 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:46.587917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676649ff-bcae-4186-8bfc-e9ff7efd6419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:46.615298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-676649ff-bcae-4186-8bfc-e9ff7efd6419 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:46.617095 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-676649ff-bcae-4186-8bfc-e9ff7efd6419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:46.653224 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997f70c-71c4-477e-a60b-897e313453bc None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:46.689407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:46.715189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 241/492] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:46.715189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd6cf2-1d21-4ec9-b8e0-3a84a12ac780 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:46.715189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd6cf2-1d21-4ec9-b8e0-3a84a12ac780 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:46.715189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd6cf2-1d21-4ec9-b8e0-3a84a12ac780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:46.730100 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ab440-50b8-4f5d-b7d0-79a873770d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:46.731231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ab440-50b8-4f5d-b7d0-79a873770d5b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:46.731775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ab440-50b8-4f5d-b7d0-79a873770d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:46.732265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ab440-50b8-4f5d-b7d0-79a873770d5b None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:46.733162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ab440-50b8-4f5d-b7d0-79a873770d5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:46.733666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ab440-50b8-4f5d-b7d0-79a873770d5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:46.735444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ab440-50b8-4f5d-b7d0-79a873770d5b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:46.751788 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-cdcd6cf2-1d21-4ec9-b8e0-3a84a12ac780 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:46.753378 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cdcd6cf2-1d21-4ec9-b8e0-3a84a12ac780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:46.760362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 252/493] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:46.761573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ab440-50b8-4f5d-b7d0-79a873770d5b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:46.887838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:46.888155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 242/494] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:46.889759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd6cf2-1d21-4ec9-b8e0-3a84a12ac780 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:46.953702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ab440-50b8-4f5d-b7d0-79a873770d5b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:47.029535 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd25238-84f9-4479-b0b0-d16f6670c076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:47.030124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd25238-84f9-4479-b0b0-d16f6670c076 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.030399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd25238-84f9-4479-b0b0-d16f6670c076 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.030668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd25238-84f9-4479-b0b0-d16f6670c076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.043496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd25238-84f9-4479-b0b0-d16f6670c076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:47.045030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd25238-84f9-4479-b0b0-d16f6670c076 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-bcd25238-84f9-4479-b0b0-d16f6670c076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:47.060821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:47.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:47.062688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:47.062688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:47.062688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:47.062688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 253/495] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:47.062913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd25238-84f9-4479-b0b0-d16f6670c076 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:47.062977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd6cf2-1d21-4ec9-b8e0-3a84a12ac780 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:47.103258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:47.127383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c3f92-c011-4b98-b2a5-d6b6fbcf3748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:47.128984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c3f92-c011-4b98-b2a5-d6b6fbcf3748 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.129378 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c3f92-c011-4b98-b2a5-d6b6fbcf3748 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.130185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c3f92-c011-4b98-b2a5-d6b6fbcf3748 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.147563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c3f92-c011-4b98-b2a5-d6b6fbcf3748 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:47.151230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c3f92-c011-4b98-b2a5-d6b6fbcf3748 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-870c3f92-c011-4b98-b2a5-d6b6fbcf3748 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:57:47.180506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:47.181612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:47.182522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:47.182522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:47.182522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:57:47.182522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:57:47.182522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:57:47.182522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:57:47.182522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:57:47.182522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:57:47.182522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:57:47.182522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:47.182522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 243/496] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Aug 24 09:57:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:47.183376 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c3f92-c011-4b98-b2a5-d6b6fbcf3748 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:47.188818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd25238-84f9-4479-b0b0-d16f6670c076 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:47.220367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:47.231212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff363b9-509d-4193-8b27-d92b2cd96e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:47.231847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff363b9-509d-4193-8b27-d92b2cd96e9f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.235567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff363b9-509d-4193-8b27-d92b2cd96e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.235567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff363b9-509d-4193-8b27-d92b2cd96e9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.244076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff363b9-509d-4193-8b27-d92b2cd96e9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-4ff363b9-509d-4193-8b27-d92b2cd96e9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:47.249052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:47.250137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:47.251144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 254/497] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:47.251981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c100b8-f3f0-4360-b6f5-3f3be3192ed1 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.251981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c100b8-f3f0-4360-b6f5-3f3be3192ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.251981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c100b8-f3f0-4360-b6f5-3f3be3192ed1 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.253211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 255/498] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:47.253913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c100b8-f3f0-4360-b6f5-3f3be3192ed1 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:47.307290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c3f92-c011-4b98-b2a5-d6b6fbcf3748 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:47.341430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61b396-c1f7-40c5-b37d-79d0e6b78a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:47.342008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61b396-c1f7-40c5-b37d-79d0e6b78a85 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.342288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61b396-c1f7-40c5-b37d-79d0e6b78a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.342562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61b396-c1f7-40c5-b37d-79d0e6b78a85 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.343212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61b396-c1f7-40c5-b37d-79d0e6b78a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:47.343523 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61b396-c1f7-40c5-b37d-79d0e6b78a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:47.344943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61b396-c1f7-40c5-b37d-79d0e6b78a85 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:47.357953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 244/499] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:47.358908 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61b396-c1f7-40c5-b37d-79d0e6b78a85 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:47.365961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c100b8-f3f0-4360-b6f5-3f3be3192ed1 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:47.401946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfa5a2-d3a3-4f7b-8cd4-2c6c84b4358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:47.402588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfa5a2-d3a3-4f7b-8cd4-2c6c84b4358b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.402867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfa5a2-d3a3-4f7b-8cd4-2c6c84b4358b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.403150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfa5a2-d3a3-4f7b-8cd4-2c6c84b4358b None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.403942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfa5a2-d3a3-4f7b-8cd4-2c6c84b4358b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:47.404308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfa5a2-d3a3-4f7b-8cd4-2c6c84b4358b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:47.405691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:47.405977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfa5a2-d3a3-4f7b-8cd4-2c6c84b4358b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:47.446624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 256/500] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Aug 24 09:57:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:47.447501 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfa5a2-d3a3-4f7b-8cd4-2c6c84b4358b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:47.466418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61b396-c1f7-40c5-b37d-79d0e6b78a85 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:47.497111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:47.502962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be9a0f-3a2d-49e6-9d75-72c5c40c9f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:47.503593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be9a0f-3a2d-49e6-9d75-72c5c40c9f78 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.503872 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be9a0f-3a2d-49e6-9d75-72c5c40c9f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.504145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be9a0f-3a2d-49e6-9d75-72c5c40c9f78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.515040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be9a0f-3a2d-49e6-9d75-72c5c40c9f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-d7be9a0f-3a2d-49e6-9d75-72c5c40c9f78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:47.520017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:47.521488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 245/501] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:47.523041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97a9cf-bc59-4f2a-9882-afd74d0c2bd9 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.524398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97a9cf-bc59-4f2a-9882-afd74d0c2bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.524398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97a9cf-bc59-4f2a-9882-afd74d0c2bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.543355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-fe97a9cf-bc59-4f2a-9882-afd74d0c2bd9 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:47.544628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-fe97a9cf-bc59-4f2a-9882-afd74d0c2bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:47.561227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfa5a2-d3a3-4f7b-8cd4-2c6c84b4358b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:47.608486 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063131f-c23e-47f7-9e28-67b9459ada4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:47.609128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063131f-c23e-47f7-9e28-67b9459ada4b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.609417 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063131f-c23e-47f7-9e28-67b9459ada4b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.609762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063131f-c23e-47f7-9e28-67b9459ada4b None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.610428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063131f-c23e-47f7-9e28-67b9459ada4b None admin] RBAC: Adding query filter params (name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:47.610730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063131f-c23e-47f7-9e28-67b9459ada4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:47.612134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063131f-c23e-47f7-9e28-67b9459ada4b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:47.621251 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 246/502] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:47.622105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97a9cf-bc59-4f2a-9882-afd74d0c2bd9 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:47.630727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 257/503] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:57:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:47.632143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063131f-c23e-47f7-9e28-67b9459ada4b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:47.656740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:47.737076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97a9cf-bc59-4f2a-9882-afd74d0c2bd9 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:47.752314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063131f-c23e-47f7-9e28-67b9459ada4b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:47.761990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:47.780482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd3c35-2e88-4af3-83cb-d69aff9853d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:47.781189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd3c35-2e88-4af3-83cb-d69aff9853d1 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.781472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd3c35-2e88-4af3-83cb-d69aff9853d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.784717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd3c35-2e88-4af3-83cb-d69aff9853d1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.790815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd3c35-2e88-4af3-83cb-d69aff9853d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:47.792553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd3c35-2e88-4af3-83cb-d69aff9853d1 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:47.794057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f778152-bd86-42a7-ae68-fa67fdf2430d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:47.794867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f778152-bd86-42a7-ae68-fa67fdf2430d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.795352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f778152-bd86-42a7-ae68-fa67fdf2430d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.795636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f778152-bd86-42a7-ae68-fa67fdf2430d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-ebbd3c35-2e88-4af3-83cb-d69aff9853d1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:57:47.805926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:47.806937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:47.807860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:47.807860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:47.807860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:57:47.807860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:57:47.807860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:57:47.807860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:57:47.807860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:57:47.807860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:57:47.807860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:57:47.807860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:47.807860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 247/504] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:47.808428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd3c35-2e88-4af3-83cb-d69aff9853d1 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:47.876701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f778152-bd86-42a7-ae68-fa67fdf2430d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:47.878742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f778152-bd86-42a7-ae68-fa67fdf2430d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:47.930977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:47.940051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd3c35-2e88-4af3-83cb-d69aff9853d1 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:47.979072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bbbcd-e76a-493d-b0bd-a0ab32c60008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:47.979674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbbcd-e76a-493d-b0bd-a0ab32c60008 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:47.979952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbbcd-e76a-493d-b0bd-a0ab32c60008 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:47.980225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbbcd-e76a-493d-b0bd-a0ab32c60008 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:47.981137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bbbcd-e76a-493d-b0bd-a0ab32c60008 None admin] RBAC: Adding query filter params (name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:47.981479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bbbcd-e76a-493d-b0bd-a0ab32c60008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:47.982937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bbbcd-e76a-493d-b0bd-a0ab32c60008 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:47.991298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 248/505] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:57:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:47.992164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bbbcd-e76a-493d-b0bd-a0ab32c60008 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:48.000932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 258/506] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:48.000932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f778152-bd86-42a7-ae68-fa67fdf2430d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:48.007936 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:48.103656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bbbcd-e76a-493d-b0bd-a0ab32c60008 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:48.113903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f778152-bd86-42a7-ae68-fa67fdf2430d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:48.137243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ac0b0-846d-46ea-9fbb-09a1960a67d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:48.137794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ac0b0-846d-46ea-9fbb-09a1960a67d8 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:48.138072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ac0b0-846d-46ea-9fbb-09a1960a67d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:48.138343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ac0b0-846d-46ea-9fbb-09a1960a67d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:48.149721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ac0b0-846d-46ea-9fbb-09a1960a67d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:48.150714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b1d0d-8369-4bd1-a9cf-0fbff56eaf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:48.151182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ac0b0-846d-46ea-9fbb-09a1960a67d8 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:48.151344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b1d0d-8369-4bd1-a9cf-0fbff56eaf4f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:48.151628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b1d0d-8369-4bd1-a9cf-0fbff56eaf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:48.151906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b1d0d-8369-4bd1-a9cf-0fbff56eaf4f None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:48.152633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b1d0d-8369-4bd1-a9cf-0fbff56eaf4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:48.152999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b1d0d-8369-4bd1-a9cf-0fbff56eaf4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:48.154469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b1d0d-8369-4bd1-a9cf-0fbff56eaf4f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-795ac0b0-846d-46ea-9fbb-09a1960a67d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:48.167010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:48.167985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:48.169143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:48.169143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:48.169143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:48.169143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 249/507] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:48.169578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ac0b0-846d-46ea-9fbb-09a1960a67d8 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:48.173566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:48.177146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 259/508] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Aug 24 09:57:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:48.178022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b1d0d-8369-4bd1-a9cf-0fbff56eaf4f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:48.247486 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:48.280449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ac0b0-846d-46ea-9fbb-09a1960a67d8 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:48.288825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b1d0d-8369-4bd1-a9cf-0fbff56eaf4f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:48.318248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d268f-e392-4e28-8382-2dfa00531a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:48.319081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d268f-e392-4e28-8382-2dfa00531a53 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:48.319366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d268f-e392-4e28-8382-2dfa00531a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:48.319645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d268f-e392-4e28-8382-2dfa00531a53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:48.329768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e757b36-6781-4f73-bc6d-64a5182c43e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:48.330212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d268f-e392-4e28-8382-2dfa00531a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:48.330385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e757b36-6781-4f73-bc6d-64a5182c43e0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:48.330662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e757b36-6781-4f73-bc6d-64a5182c43e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:48.330930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e757b36-6781-4f73-bc6d-64a5182c43e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:48.331667 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d268f-e392-4e28-8382-2dfa00531a53 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:48.341708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e757b36-6781-4f73-bc6d-64a5182c43e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:48.343148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e757b36-6781-4f73-bc6d-64a5182c43e0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-736d268f-e392-4e28-8382-2dfa00531a53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:48.346413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:48.347440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:48.348405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:48.348405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:48.348405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:48.348405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 250/509] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:48.348681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d268f-e392-4e28-8382-2dfa00531a53 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-2e757b36-6781-4f73-bc6d-64a5182c43e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:48.359611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:48.360602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:48.362584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:48.362584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:48.362584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:48.362584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 260/510] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:48.363323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e757b36-6781-4f73-bc6d-64a5182c43e0 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:48.410225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:48.459593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d268f-e392-4e28-8382-2dfa00531a53 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:48.469205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e757b36-6781-4f73-bc6d-64a5182c43e0 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:48.488300 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:48.496100 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c380e4-577c-498c-9ad3-57fbf9577693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:48.496718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c380e4-577c-498c-9ad3-57fbf9577693 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:48.497027 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c380e4-577c-498c-9ad3-57fbf9577693 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:48.497298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c380e4-577c-498c-9ad3-57fbf9577693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:48.506681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83a485-cdc6-4fd7-8b4d-c4ff7b180921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:48.507275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83a485-cdc6-4fd7-8b4d-c4ff7b180921 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:48.533339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83a485-cdc6-4fd7-8b4d-c4ff7b180921 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:48.533339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83a485-cdc6-4fd7-8b4d-c4ff7b180921 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:48.533537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c380e4-577c-498c-9ad3-57fbf9577693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:48.533611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83a485-cdc6-4fd7-8b4d-c4ff7b180921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:48.533611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83a485-cdc6-4fd7-8b4d-c4ff7b180921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:48.533731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c380e4-577c-498c-9ad3-57fbf9577693 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:48.533826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83a485-cdc6-4fd7-8b4d-c4ff7b180921 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:48.533826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 261/511] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:48.533826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83a485-cdc6-4fd7-8b4d-c4ff7b180921 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-52c380e4-577c-498c-9ad3-57fbf9577693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:48.534012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:48.534813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:48.535654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:48.535654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:48.535654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:48.535654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 251/512] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:48.535654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c380e4-577c-498c-9ad3-57fbf9577693 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:48.629462 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83a485-cdc6-4fd7-8b4d-c4ff7b180921 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:48.636479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c380e4-577c-498c-9ad3-57fbf9577693 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:48.645991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:48.669671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a0e17-2cfa-4d6a-b8ae-1dcd1b24bb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:48.670171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0e17-2cfa-4d6a-b8ae-1dcd1b24bb0f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:48.670451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0e17-2cfa-4d6a-b8ae-1dcd1b24bb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:48.670725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0e17-2cfa-4d6a-b8ae-1dcd1b24bb0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:48.671268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f70cf7-96fb-4e3a-b69c-227d19ad1628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:48.671861 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f70cf7-96fb-4e3a-b69c-227d19ad1628 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:48.672136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f70cf7-96fb-4e3a-b69c-227d19ad1628 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:48.672405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f70cf7-96fb-4e3a-b69c-227d19ad1628 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:48.673078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f70cf7-96fb-4e3a-b69c-227d19ad1628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:48.673434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f70cf7-96fb-4e3a-b69c-227d19ad1628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:48.674769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f70cf7-96fb-4e3a-b69c-227d19ad1628 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:48.687344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 252/513] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:48.688182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f70cf7-96fb-4e3a-b69c-227d19ad1628 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:48.724838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:48.748253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a0e17-2cfa-4d6a-b8ae-1dcd1b24bb0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:48.749765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a0e17-2cfa-4d6a-b8ae-1dcd1b24bb0f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:48.796805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f70cf7-96fb-4e3a-b69c-227d19ad1628 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:48.832796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6105a25-eb11-4f5f-a52d-7223e4544c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:48.834455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6105a25-eb11-4f5f-a52d-7223e4544c9b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:48.834741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6105a25-eb11-4f5f-a52d-7223e4544c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:48.835009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6105a25-eb11-4f5f-a52d-7223e4544c9b None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:48.835661 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6105a25-eb11-4f5f-a52d-7223e4544c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:48.835964 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6105a25-eb11-4f5f-a52d-7223e4544c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:48.837377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6105a25-eb11-4f5f-a52d-7223e4544c9b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:48.852524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 253/514] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:48.853420 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6105a25-eb11-4f5f-a52d-7223e4544c9b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:48.857265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 262/515] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:48.858233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a0e17-2cfa-4d6a-b8ae-1dcd1b24bb0f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:48.887983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:48.938738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:48.962859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a0e17-2cfa-4d6a-b8ae-1dcd1b24bb0f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:48.963776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6105a25-eb11-4f5f-a52d-7223e4544c9b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:48.997220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca13934-c920-4fb8-b382-a916f63c29fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:48.997820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca13934-c920-4fb8-b382-a916f63c29fe None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:48.998151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca13934-c920-4fb8-b382-a916f63c29fe None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:48.998449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2d37d-1d10-4f83-8fc9-2664463b2664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:48.998580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca13934-c920-4fb8-b382-a916f63c29fe None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:48.999043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2d37d-1d10-4f83-8fc9-2664463b2664 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:48.999136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca13934-c920-4fb8-b382-a916f63c29fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:48.999340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2d37d-1d10-4f83-8fc9-2664463b2664 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:48.999456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca13934-c920-4fb8-b382-a916f63c29fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:48.999597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2d37d-1d10-4f83-8fc9-2664463b2664 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:49.000714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca13934-c920-4fb8-b382-a916f63c29fe None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:49.008559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2d37d-1d10-4f83-8fc9-2664463b2664 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:49.010108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2d37d-1d10-4f83-8fc9-2664463b2664 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:49.013930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 263/516] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:49.014746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca13934-c920-4fb8-b382-a916f63c29fe None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b9c2d37d-1d10-4f83-8fc9-2664463b2664 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:57:49.022438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:49.023391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:49.024324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:49.024324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:49.024324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:57:49.024324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:57:49.024324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:57:49.024324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:57:49.024324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:57:49.024324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:57:49.024324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:57:49.024324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:49.024324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 254/517] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:49.025764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2d37d-1d10-4f83-8fc9-2664463b2664 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:49.114610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:49.119659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca13934-c920-4fb8-b382-a916f63c29fe None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:49.126591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2d37d-1d10-4f83-8fc9-2664463b2664 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:49.155184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106562f-d467-484f-8164-e7254c6a3025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:49.155885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106562f-d467-484f-8164-e7254c6a3025 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:49.156308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106562f-d467-484f-8164-e7254c6a3025 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:49.156645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106562f-d467-484f-8164-e7254c6a3025 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:49.157552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106562f-d467-484f-8164-e7254c6a3025 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:49.159270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106562f-d467-484f-8164-e7254c6a3025 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:49.161944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed4312-8481-42be-95e1-2df8020b45c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:49.162572 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4312-8481-42be-95e1-2df8020b45c8 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:49.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4312-8481-42be-95e1-2df8020b45c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:49.163384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4312-8481-42be-95e1-2df8020b45c8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:49.169878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:49.173437 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed4312-8481-42be-95e1-2df8020b45c8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:49.174942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed4312-8481-42be-95e1-2df8020b45c8 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-20ed4312-8481-42be-95e1-2df8020b45c8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:57:49.186571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:49.187643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 255/518] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:49.188988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed4312-8481-42be-95e1-2df8020b45c8 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:49.290266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed4312-8481-42be-95e1-2df8020b45c8 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:49.303732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 264/519] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:57:49 2022] POST /identity/v3/users => generated 247 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:49.304607 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106562f-d467-484f-8164-e7254c6a3025 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:49.331552 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef8460-577c-4855-9dcf-c7056fc4604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:49.332331 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef8460-577c-4855-9dcf-c7056fc4604d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:49.332461 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef8460-577c-4855-9dcf-c7056fc4604d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:49.332759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef8460-577c-4855-9dcf-c7056fc4604d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:49.346886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef8460-577c-4855-9dcf-c7056fc4604d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:49.348520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef8460-577c-4855-9dcf-c7056fc4604d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:49.359508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-9bef8460-577c-4855-9dcf-c7056fc4604d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:49.367501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:49.368622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:49.369671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:49.369671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:49.369671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:49.371291 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 256/520] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:49.372335 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef8460-577c-4855-9dcf-c7056fc4604d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:49.440886 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106562f-d467-484f-8164-e7254c6a3025 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:49.481935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcb8fc-3f29-4a2d-991b-eaf8d4ce06f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:49.482565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcb8fc-3f29-4a2d-991b-eaf8d4ce06f8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:49.482846 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcb8fc-3f29-4a2d-991b-eaf8d4ce06f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:49.483139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcb8fc-3f29-4a2d-991b-eaf8d4ce06f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:49.492801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcb8fc-3f29-4a2d-991b-eaf8d4ce06f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:49.494972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcb8fc-3f29-4a2d-991b-eaf8d4ce06f8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:49.499136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef8460-577c-4855-9dcf-c7056fc4604d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-69bcb8fc-3f29-4a2d-991b-eaf8d4ce06f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:57:49.509653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:49.510731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:49.511609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:57:49.511609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:57:49.511609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:57:49.511609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:57:49.511609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:57:49.511609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:57:49.511609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:49.511609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:49.511609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 265/521] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:49.512090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcb8fc-3f29-4a2d-991b-eaf8d4ce06f8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:49.544021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47630e-08a0-4566-abcc-779805609b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:49.544929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47630e-08a0-4566-abcc-779805609b09 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:49.545305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47630e-08a0-4566-abcc-779805609b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:49.545626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47630e-08a0-4566-abcc-779805609b09 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:49.546516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47630e-08a0-4566-abcc-779805609b09 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:49.547161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47630e-08a0-4566-abcc-779805609b09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:49.549066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47630e-08a0-4566-abcc-779805609b09 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:49.570973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:49.595000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 257/522] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Aug 24 09:57:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:49.596199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47630e-08a0-4566-abcc-779805609b09 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:49.625517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcb8fc-3f29-4a2d-991b-eaf8d4ce06f8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:49.659641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:49.665376 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1271a-9204-48ae-a013-1539fedec302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:49.665993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1271a-9204-48ae-a013-1539fedec302 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:49.666277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1271a-9204-48ae-a013-1539fedec302 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:49.666554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1271a-9204-48ae-a013-1539fedec302 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:49.667320 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1271a-9204-48ae-a013-1539fedec302 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:49.667777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1271a-9204-48ae-a013-1539fedec302 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:49.669287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1271a-9204-48ae-a013-1539fedec302 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:49.688309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 266/523] 10.222.0.30 () {56 vars in 1139 bytes} [Wed Aug 24 09:57:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:49.689339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1271a-9204-48ae-a013-1539fedec302 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:49.706841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47630e-08a0-4566-abcc-779805609b09 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:49.747479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d47f6-0187-4970-9350-304299122d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:49.748410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d47f6-0187-4970-9350-304299122d67 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:49.748682 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d47f6-0187-4970-9350-304299122d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:49.748987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d47f6-0187-4970-9350-304299122d67 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:49.749645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d47f6-0187-4970-9350-304299122d67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:49.749950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d47f6-0187-4970-9350-304299122d67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:49.751812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d47f6-0187-4970-9350-304299122d67 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:49.768377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 258/524] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:49.769425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d47f6-0187-4970-9350-304299122d67 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:49.810850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1271a-9204-48ae-a013-1539fedec302 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:49.823121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:49.848988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48616049-dfa4-46ae-bf06-f6cd594991af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:49.849700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48616049-dfa4-46ae-bf06-f6cd594991af None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:49.849980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48616049-dfa4-46ae-bf06-f6cd594991af None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:49.850252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48616049-dfa4-46ae-bf06-f6cd594991af None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:49.850948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48616049-dfa4-46ae-bf06-f6cd594991af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:49.851258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48616049-dfa4-46ae-bf06-f6cd594991af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:49.852961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48616049-dfa4-46ae-bf06-f6cd594991af None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:49.871833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 267/525] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:57:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:49.874077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48616049-dfa4-46ae-bf06-f6cd594991af None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:49.881130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d47f6-0187-4970-9350-304299122d67 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:49.901260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:49.920826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243d7a4-ff74-48e6-8fa8-759eb69251e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:49.921635 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243d7a4-ff74-48e6-8fa8-759eb69251e4 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:49.922244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243d7a4-ff74-48e6-8fa8-759eb69251e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:49.922527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243d7a4-ff74-48e6-8fa8-759eb69251e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:49.981554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48616049-dfa4-46ae-bf06-f6cd594991af None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:50.003448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243d7a4-ff74-48e6-8fa8-759eb69251e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:50.004976 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243d7a4-ff74-48e6-8fa8-759eb69251e4 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:50.018637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac543675-3833-4255-b149-1e50b9438151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:50.019485 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac543675-3833-4255-b149-1e50b9438151 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:50.019769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac543675-3833-4255-b149-1e50b9438151 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:50.020046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac543675-3833-4255-b149-1e50b9438151 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:50.064852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:50.094927 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac543675-3833-4255-b149-1e50b9438151 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:50.097510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac543675-3833-4255-b149-1e50b9438151 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:50.117764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 259/526] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:50.117764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243d7a4-ff74-48e6-8fa8-759eb69251e4 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:50.134455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:50.222005 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243d7a4-ff74-48e6-8fa8-759eb69251e4 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:50.225171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 268/527] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:57:50.226220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac543675-3833-4255-b149-1e50b9438151 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:50.268771 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d36627-a0dd-4fd0-bb67-e534e3e51796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:50.269388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d36627-a0dd-4fd0-bb67-e534e3e51796 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:50.269669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d36627-a0dd-4fd0-bb67-e534e3e51796 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:50.269946 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d36627-a0dd-4fd0-bb67-e534e3e51796 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:50.278041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d36627-a0dd-4fd0-bb67-e534e3e51796 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:50.279493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d36627-a0dd-4fd0-bb67-e534e3e51796 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-60d36627-a0dd-4fd0-bb67-e534e3e51796 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:57:50.293354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:50.299136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:50.300142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:57:50.300142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:57:50.300142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:57:50.300142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:57:50.300142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:57:50.300142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:57:50.300142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:50.300142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:50.300142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 260/528] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:50.300142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d36627-a0dd-4fd0-bb67-e534e3e51796 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:50.304666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:50.341087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac543675-3833-4255-b149-1e50b9438151 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:50.383163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29e893-658c-437a-adc3-2d703feb2edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:50.383945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29e893-658c-437a-adc3-2d703feb2edd None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:50.384426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29e893-658c-437a-adc3-2d703feb2edd None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:50.384947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29e893-658c-437a-adc3-2d703feb2edd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:50.385890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29e893-658c-437a-adc3-2d703feb2edd None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=7cd6d2291f934fbdb25c410c2c3abcb6) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:50.386230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29e893-658c-437a-adc3-2d703feb2edd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:50.387786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29e893-658c-437a-adc3-2d703feb2edd None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:50.392563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:50.394453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 269/529] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:50 2022] GET /identity/v3/role_assignments?user.id=7cd6d2291f934fbdb25c410c2c3abcb6&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:50.394453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29e893-658c-437a-adc3-2d703feb2edd None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:50.435097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d36627-a0dd-4fd0-bb67-e534e3e51796 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:50.485880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79626-d553-4a60-ac29-faa4c56f2acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:50.487085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79626-d553-4a60-ac29-faa4c56f2acc None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:50.487413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79626-d553-4a60-ac29-faa4c56f2acc None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:50.487689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79626-d553-4a60-ac29-faa4c56f2acc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:50.504335 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f79626-d553-4a60-ac29-faa4c56f2acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:50.507527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f79626-d553-4a60-ac29-faa4c56f2acc None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-f9f79626-d553-4a60-ac29-faa4c56f2acc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:50.527301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:50.528652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:50.529612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:50.529612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:50.529612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:50.529612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 261/530] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:50.530271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79626-d553-4a60-ac29-faa4c56f2acc None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:50.556720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29e893-658c-437a-adc3-2d703feb2edd None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:50.602580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adc1f5-62f0-4ff5-a47e-cb4b759510dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:50.603669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc1f5-62f0-4ff5-a47e-cb4b759510dc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:50.604151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc1f5-62f0-4ff5-a47e-cb4b759510dc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:50.604619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc1f5-62f0-4ff5-a47e-cb4b759510dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:50.607143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:50.618139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adc1f5-62f0-4ff5-a47e-cb4b759510dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:50.619882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adc1f5-62f0-4ff5-a47e-cb4b759510dc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-c6adc1f5-62f0-4ff5-a47e-cb4b759510dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:50.636432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:50.643544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:50.644386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:50.644386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:50.644386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:50.645071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 270/531] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:50.647188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adc1f5-62f0-4ff5-a47e-cb4b759510dc None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:50.653741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79626-d553-4a60-ac29-faa4c56f2acc None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:50.684144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:50.690469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0224a27-0bb1-4d3e-ba4f-3203fd4fb35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:50.694781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0224a27-0bb1-4d3e-ba4f-3203fd4fb35e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:50.694781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0224a27-0bb1-4d3e-ba4f-3203fd4fb35e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:50.694781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0224a27-0bb1-4d3e-ba4f-3203fd4fb35e None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:50.694781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0224a27-0bb1-4d3e-ba4f-3203fd4fb35e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:50.694781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0224a27-0bb1-4d3e-ba4f-3203fd4fb35e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:50.695207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0224a27-0bb1-4d3e-ba4f-3203fd4fb35e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:50.715490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 262/532] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:57:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:50.716438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0224a27-0bb1-4d3e-ba4f-3203fd4fb35e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:50.766150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adc1f5-62f0-4ff5-a47e-cb4b759510dc None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:50.807445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193d369-ab04-4050-af34-7b21a6b9e354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:50.808504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193d369-ab04-4050-af34-7b21a6b9e354 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:50.809031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193d369-ab04-4050-af34-7b21a6b9e354 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:50.809553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193d369-ab04-4050-af34-7b21a6b9e354 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:50.810448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193d369-ab04-4050-af34-7b21a6b9e354 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:50.810968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193d369-ab04-4050-af34-7b21a6b9e354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:50.812657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193d369-ab04-4050-af34-7b21a6b9e354 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:50.825288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0224a27-0bb1-4d3e-ba4f-3203fd4fb35e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:50.832713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 271/533] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:50.834449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193d369-ab04-4050-af34-7b21a6b9e354 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:50.863916 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29be1d-fc7b-437f-b26d-e5bc1df2a17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:50.865351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29be1d-fc7b-437f-b26d-e5bc1df2a17b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:50.867061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29be1d-fc7b-437f-b26d-e5bc1df2a17b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:50.867483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29be1d-fc7b-437f-b26d-e5bc1df2a17b None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:50.868378 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29be1d-fc7b-437f-b26d-e5bc1df2a17b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:50.869086 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29be1d-fc7b-437f-b26d-e5bc1df2a17b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:50.871814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29be1d-fc7b-437f-b26d-e5bc1df2a17b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:50.888116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 263/534] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:50.889995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29be1d-fc7b-437f-b26d-e5bc1df2a17b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:50.901299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:50.956375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:50.956953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193d369-ab04-4050-af34-7b21a6b9e354 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:50.996761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a819fe-1f4b-4cfb-a80a-5888e51ebd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:50.997638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a819fe-1f4b-4cfb-a80a-5888e51ebd22 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:50.998018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a819fe-1f4b-4cfb-a80a-5888e51ebd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:50.998719 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a819fe-1f4b-4cfb-a80a-5888e51ebd22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:50.999364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a819fe-1f4b-4cfb-a80a-5888e51ebd22 None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=d0b124ca267b4a4391557ad1396524cf) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:50.999847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a819fe-1f4b-4cfb-a80a-5888e51ebd22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:51.001471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a819fe-1f4b-4cfb-a80a-5888e51ebd22 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:51.007752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 272/535] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:50 2022] GET /identity/v3/role_assignments?user.id=d0b124ca267b4a4391557ad1396524cf&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:51.008700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a819fe-1f4b-4cfb-a80a-5888e51ebd22 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:51.031907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29be1d-fc7b-437f-b26d-e5bc1df2a17b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:51.114174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84b5bb-346f-4860-85ed-6ddd5dc52fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:51.114841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84b5bb-346f-4860-85ed-6ddd5dc52fdf None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:51.115121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84b5bb-346f-4860-85ed-6ddd5dc52fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:51.115400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84b5bb-346f-4860-85ed-6ddd5dc52fdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:51.127055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84b5bb-346f-4860-85ed-6ddd5dc52fdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:51.128565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84b5bb-346f-4860-85ed-6ddd5dc52fdf None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-7c84b5bb-346f-4860-85ed-6ddd5dc52fdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:57:51.157954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:51.158874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:51.159700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:57:51.159700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:57:51.159700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:57:51.159700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:57:51.159700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:57:51.159700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:57:51.159700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:51.159700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:51.163130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 264/536] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:51.167079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84b5bb-346f-4860-85ed-6ddd5dc52fdf None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:51.228037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a819fe-1f4b-4cfb-a80a-5888e51ebd22 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:51.273284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:51.288468 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36351feb-1237-4f4b-a721-8c2cc65971b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:51.289391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36351feb-1237-4f4b-a721-8c2cc65971b8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:51.289887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36351feb-1237-4f4b-a721-8c2cc65971b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:51.290302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36351feb-1237-4f4b-a721-8c2cc65971b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:51.299996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36351feb-1237-4f4b-a721-8c2cc65971b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:51.302844 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36351feb-1237-4f4b-a721-8c2cc65971b8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-36351feb-1237-4f4b-a721-8c2cc65971b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:57:51.327170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:51.339643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:51.340570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:57:51.340570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:57:51.340570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:57:51.340570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:57:51.340570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:57:51.340570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:57:51.340570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:51.340570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:51.340570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 273/537] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:51.340570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36351feb-1237-4f4b-a721-8c2cc65971b8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:51.363289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84b5bb-346f-4860-85ed-6ddd5dc52fdf None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:51.379624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:51.412375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9190e2-a885-4f17-a13f-84e12ee45dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:51.413308 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9190e2-a885-4f17-a13f-84e12ee45dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:51.413565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9190e2-a885-4f17-a13f-84e12ee45dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:51.413817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9190e2-a885-4f17-a13f-84e12ee45dc3 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:51.414527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9190e2-a885-4f17-a13f-84e12ee45dc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:51.414833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9190e2-a885-4f17-a13f-84e12ee45dc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:51.417159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9190e2-a885-4f17-a13f-84e12ee45dc3 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:51.437286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 265/538] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:57:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:51.439407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603f213-c715-4a95-b085-0fb39d8e7994 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:51.440169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603f213-c715-4a95-b085-0fb39d8e7994 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:51.441055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603f213-c715-4a95-b085-0fb39d8e7994 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:51.444082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 266/539] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:51.444971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603f213-c715-4a95-b085-0fb39d8e7994 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:51.483694 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36351feb-1237-4f4b-a721-8c2cc65971b8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:51.531077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d5b5e-4e08-459b-88fe-a411fa840976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:51.532746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d5b5e-4e08-459b-88fe-a411fa840976 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:51.533209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d5b5e-4e08-459b-88fe-a411fa840976 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:51.533646 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d5b5e-4e08-459b-88fe-a411fa840976 None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:51.534492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d5b5e-4e08-459b-88fe-a411fa840976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:51.535310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d5b5e-4e08-459b-88fe-a411fa840976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:51.538457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d5b5e-4e08-459b-88fe-a411fa840976 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:51.556479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 274/540] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:57:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:51.558369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576de0c-c5a5-4114-9b34-cbcf65d3077c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:51.558844 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576de0c-c5a5-4114-9b34-cbcf65d3077c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:51.559614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576de0c-c5a5-4114-9b34-cbcf65d3077c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:51.573123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603f213-c715-4a95-b085-0fb39d8e7994 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:51.585067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-d576de0c-c5a5-4114-9b34-cbcf65d3077c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:51.588121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d576de0c-c5a5-4114-9b34-cbcf65d3077c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:51.619064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9e50a-64ea-475f-bcef-8d3861f54ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:51.621254 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9e50a-64ea-475f-bcef-8d3861f54ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:51.622373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:51.623494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9e50a-64ea-475f-bcef-8d3861f54ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:51.623861 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9e50a-64ea-475f-bcef-8d3861f54ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:51.624690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9e50a-64ea-475f-bcef-8d3861f54ed4 None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=c71783d1327a47149a74414218dd7219) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:51.625087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9e50a-64ea-475f-bcef-8d3861f54ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:51.626610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9e50a-64ea-475f-bcef-8d3861f54ed4 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:51.644621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 267/541] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:51 2022] GET /identity/v3/role_assignments?user.id=c71783d1327a47149a74414218dd7219&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:51.644621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9e50a-64ea-475f-bcef-8d3861f54ed4 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:51.680889 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 275/542] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:51.708836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:51.839550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9e50a-64ea-475f-bcef-8d3861f54ed4 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:51.878938 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90173e7-c289-4652-8120-a1b47fdda94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:51.879549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90173e7-c289-4652-8120-a1b47fdda94f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:51.880344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90173e7-c289-4652-8120-a1b47fdda94f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:51.880743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90173e7-c289-4652-8120-a1b47fdda94f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:51.882098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90173e7-c289-4652-8120-a1b47fdda94f None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=ae9b6525009b41288dd918437acee92a) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:51.882412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90173e7-c289-4652-8120-a1b47fdda94f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:51.883863 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90173e7-c289-4652-8120-a1b47fdda94f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:51.890177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 268/543] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:51 2022] GET /identity/v3/role_assignments?user.id=ae9b6525009b41288dd918437acee92a&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:52.051083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576de0c-c5a5-4114-9b34-cbcf65d3077c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:52.196590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576de0c-c5a5-4114-9b34-cbcf65d3077c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:52.246628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619f617-951c-4732-a2fc-9210db2a754f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:52.247258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f617-951c-4732-a2fc-9210db2a754f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:52.247576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f617-951c-4732-a2fc-9210db2a754f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:52.247834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f617-951c-4732-a2fc-9210db2a754f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:52.269619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619f617-951c-4732-a2fc-9210db2a754f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-7619f617-951c-4732-a2fc-9210db2a754f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:52.275080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:52.276079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:52.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 276/544] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:52.284635 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90173e7-c289-4652-8120-a1b47fdda94f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:52.444901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:52.448715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90173e7-c289-4652-8120-a1b47fdda94f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:52.495230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560fe65-030a-4312-b9bb-fdd1e23a6a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:52.495845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560fe65-030a-4312-b9bb-fdd1e23a6a0d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:52.496128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560fe65-030a-4312-b9bb-fdd1e23a6a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:52.496406 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560fe65-030a-4312-b9bb-fdd1e23a6a0d None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:52.497109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560fe65-030a-4312-b9bb-fdd1e23a6a0d None admin] RBAC: Adding query filter params (name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:52.497416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560fe65-030a-4312-b9bb-fdd1e23a6a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:52.498810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560fe65-030a-4312-b9bb-fdd1e23a6a0d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:52.512941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 269/545] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:57:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:52.518942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619f617-951c-4732-a2fc-9210db2a754f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:52.570179 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:52.685328 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619f617-951c-4732-a2fc-9210db2a754f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:52.714936 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c9f8d-b40f-4963-8eec-96260894e585 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:52.715116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c9f8d-b40f-4963-8eec-96260894e585 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:52.715392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c9f8d-b40f-4963-8eec-96260894e585 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:52.717018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 270/546] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:52.726021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e87217-66b9-425e-b1e6-793aa99e45c6 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:52.726290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e87217-66b9-425e-b1e6-793aa99e45c6 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:52.726565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e87217-66b9-425e-b1e6-793aa99e45c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:52.732663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb1788-2e62-439e-837b-79e24f963283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:52.733312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb1788-2e62-439e-837b-79e24f963283 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:52.733587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb1788-2e62-439e-837b-79e24f963283 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:52.733857 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb1788-2e62-439e-837b-79e24f963283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:52.753508 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb1788-2e62-439e-837b-79e24f963283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:52.755562 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb1788-2e62-439e-837b-79e24f963283 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:52.769347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b1e87217-66b9-425e-b1e6-793aa99e45c6 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:52.770689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b1e87217-66b9-425e-b1e6-793aa99e45c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-46cb1788-2e62-439e-837b-79e24f963283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:52.788803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:52.789811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:52.791163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:52.791163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:52.791163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:52.791163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 277/547] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:52.797787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb1788-2e62-439e-837b-79e24f963283 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:52.856493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 271/548] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:52.946885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:52.987102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb1788-2e62-439e-837b-79e24f963283 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:53.035819 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a0a92-611d-4abd-8be2-9e35ba0a8f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:53.037161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a0a92-611d-4abd-8be2-9e35ba0a8f44 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:53.037161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a0a92-611d-4abd-8be2-9e35ba0a8f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:53.037466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a0a92-611d-4abd-8be2-9e35ba0a8f44 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:53.038255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a0a92-611d-4abd-8be2-9e35ba0a8f44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:53.038769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a0a92-611d-4abd-8be2-9e35ba0a8f44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:53.040614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a0a92-611d-4abd-8be2-9e35ba0a8f44 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:53.063091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 278/549] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:53.069980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e87217-66b9-425e-b1e6-793aa99e45c6 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:53.184070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:53.194998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e87217-66b9-425e-b1e6-793aa99e45c6 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:53.240612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeafc06-729a-44c1-90f0-9900b37f38f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:53.241192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeafc06-729a-44c1-90f0-9900b37f38f6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:53.241472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeafc06-729a-44c1-90f0-9900b37f38f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:53.241745 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeafc06-729a-44c1-90f0-9900b37f38f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:53.249485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeafc06-729a-44c1-90f0-9900b37f38f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:53.250951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeafc06-729a-44c1-90f0-9900b37f38f6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-ceeafc06-729a-44c1-90f0-9900b37f38f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:57:53.265624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:53.266807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:53.267728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:53.267728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:53.267728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:57:53.267728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:57:53.267728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:57:53.267728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:57:53.267728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:57:53.267728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:57:53.267728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:57:53.267728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:53.267728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 272/550] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:57:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:53.280573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a0a92-611d-4abd-8be2-9e35ba0a8f44 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:53.332632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeafc06-729a-44c1-90f0-9900b37f38f6 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:53.457806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeafc06-729a-44c1-90f0-9900b37f38f6 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:53.459164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a0a92-611d-4abd-8be2-9e35ba0a8f44 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:53.507552 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ade87b-3cad-4488-87d9-2c14dcf7188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:53.508113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ade87b-3cad-4488-87d9-2c14dcf7188a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:53.508397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ade87b-3cad-4488-87d9-2c14dcf7188a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:53.508672 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ade87b-3cad-4488-87d9-2c14dcf7188a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:53.525722 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:53.534136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ade87b-3cad-4488-87d9-2c14dcf7188a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:53.539676 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a01a2a-6417-4629-a870-06f801e7c7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:53.540296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01a2a-6417-4629-a870-06f801e7c7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:53.540575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01a2a-6417-4629-a870-06f801e7c7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:53.540849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01a2a-6417-4629-a870-06f801e7c7b2 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:53.541666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a01a2a-6417-4629-a870-06f801e7c7b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:53.541974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a01a2a-6417-4629-a870-06f801e7c7b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:53.543459 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a01a2a-6417-4629-a870-06f801e7c7b2 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-a4ade87b-3cad-4488-87d9-2c14dcf7188a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:53.548508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:53.549477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:53.550305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:53.554196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 273/551] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:57:53.561941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ade87b-3cad-4488-87d9-2c14dcf7188a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:53.584560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:53.588701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 279/552] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Aug 24 09:57:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:53.595803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a01a2a-6417-4629-a870-06f801e7c7b2 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:53.726080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ade87b-3cad-4488-87d9-2c14dcf7188a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:53.764484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efebcd-53ce-4de8-9349-2959c7e6b357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:53.765051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efebcd-53ce-4de8-9349-2959c7e6b357 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:53.765334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efebcd-53ce-4de8-9349-2959c7e6b357 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:53.765606 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efebcd-53ce-4de8-9349-2959c7e6b357 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:53.766288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efebcd-53ce-4de8-9349-2959c7e6b357 None admin] RBAC: Adding query filter params (name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:53.766684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efebcd-53ce-4de8-9349-2959c7e6b357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:53.769271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efebcd-53ce-4de8-9349-2959c7e6b357 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:53.786556 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 274/553] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:57:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:53.788288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae6b63-1a20-4909-a52c-427a0b94e1cb None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:53.788578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae6b63-1a20-4909-a52c-427a0b94e1cb None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:53.788881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae6b63-1a20-4909-a52c-427a0b94e1cb None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:53.790214 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 275/554] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:53.790977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae6b63-1a20-4909-a52c-427a0b94e1cb None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:53.816312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a01a2a-6417-4629-a870-06f801e7c7b2 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:53.853955 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814c894-6bd1-46c8-bf3e-cb2bd1a0c9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:53.854671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814c894-6bd1-46c8-bf3e-cb2bd1a0c9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:53.854962 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814c894-6bd1-46c8-bf3e-cb2bd1a0c9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:53.855236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814c894-6bd1-46c8-bf3e-cb2bd1a0c9f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:53.889230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:53.923904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:53.931431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae6b63-1a20-4909-a52c-427a0b94e1cb None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:53.938534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814c894-6bd1-46c8-bf3e-cb2bd1a0c9f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:53.940263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814c894-6bd1-46c8-bf3e-cb2bd1a0c9f0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:53.986703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19409d-2fd4-48df-9072-c3ec676ea725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:53.988489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19409d-2fd4-48df-9072-c3ec676ea725 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:53.989703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19409d-2fd4-48df-9072-c3ec676ea725 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:53.995488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19409d-2fd4-48df-9072-c3ec676ea725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.010452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19409d-2fd4-48df-9072-c3ec676ea725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:54.011923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19409d-2fd4-48df-9072-c3ec676ea725 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-7c19409d-2fd4-48df-9072-c3ec676ea725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:54.027511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:54.028565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:54.029490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:54.029490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:54.029490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:54.033197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 276/555] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:54.034515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3feea-d0eb-4e5c-a707-9c4cfea77ffd None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.034832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3feea-d0eb-4e5c-a707-9c4cfea77ffd None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.035219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3feea-d0eb-4e5c-a707-9c4cfea77ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.057191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-86b3feea-d0eb-4e5c-a707-9c4cfea77ffd None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:54.058610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-86b3feea-d0eb-4e5c-a707-9c4cfea77ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:54.088542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 280/556] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:57:54.088542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814c894-6bd1-46c8-bf3e-cb2bd1a0c9f0 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:54.150204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 277/557] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:54.151605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfe89f-3146-455a-80ac-f867e6050c03 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.151891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfe89f-3146-455a-80ac-f867e6050c03 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.152167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfe89f-3146-455a-80ac-f867e6050c03 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.153568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 278/558] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:54.158215 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfe89f-3146-455a-80ac-f867e6050c03 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:54.199698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:54.211172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814c894-6bd1-46c8-bf3e-cb2bd1a0c9f0 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:54.236702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:54.247602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbe258-50c5-4c45-bbac-99b1bb814f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:54.248214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbe258-50c5-4c45-bbac-99b1bb814f94 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.248496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbe258-50c5-4c45-bbac-99b1bb814f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.248775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbe258-50c5-4c45-bbac-99b1bb814f94 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.249471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbe258-50c5-4c45-bbac-99b1bb814f94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:54.250182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbe258-50c5-4c45-bbac-99b1bb814f94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:54.253327 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbe258-50c5-4c45-bbac-99b1bb814f94 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:54.274445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 281/559] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:54.275785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cc673-aaa9-42a9-b9f2-4fd6e780e6cf None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.276082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cc673-aaa9-42a9-b9f2-4fd6e780e6cf None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.276364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cc673-aaa9-42a9-b9f2-4fd6e780e6cf None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.281677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 282/560] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:54.281677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9fc42-0901-401f-80d2-d588c8a0707f None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.281677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9fc42-0901-401f-80d2-d588c8a0707f None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.281677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9fc42-0901-401f-80d2-d588c8a0707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.297195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfe89f-3146-455a-80ac-f867e6050c03 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:54.300743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-34f9fc42-0901-401f-80d2-d588c8a0707f None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:54.302899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-34f9fc42-0901-401f-80d2-d588c8a0707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:54.343274 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07796255-95d9-4546-8c1f-ee0f0bb07d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:54.344279 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07796255-95d9-4546-8c1f-ee0f0bb07d6f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.344714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07796255-95d9-4546-8c1f-ee0f0bb07d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.345147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07796255-95d9-4546-8c1f-ee0f0bb07d6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.361585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07796255-95d9-4546-8c1f-ee0f0bb07d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:54.363685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07796255-95d9-4546-8c1f-ee0f0bb07d6f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-07796255-95d9-4546-8c1f-ee0f0bb07d6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:54.379886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:54.380988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:54.381964 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:54.381964 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:54.381964 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:54.381964 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 279/561] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:54.382797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07796255-95d9-4546-8c1f-ee0f0bb07d6f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:54.396702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 283/562] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:54.396702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef91c20-b046-4650-b47c-29de982a8f7a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.396702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef91c20-b046-4650-b47c-29de982a8f7a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.396702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef91c20-b046-4650-b47c-29de982a8f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.414360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-3ef91c20-b046-4650-b47c-29de982a8f7a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:54.415667 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3ef91c20-b046-4650-b47c-29de982a8f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:54.493191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 284/563] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:54.494295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef91c20-b046-4650-b47c-29de982a8f7a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:54.506994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07796255-95d9-4546-8c1f-ee0f0bb07d6f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:54.514099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:54.517399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:54.552128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcb272-6d82-491c-b1dc-2e7b61073ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:54.553310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcb272-6d82-491c-b1dc-2e7b61073ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.553775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcb272-6d82-491c-b1dc-2e7b61073ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.554228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcb272-6d82-491c-b1dc-2e7b61073ab7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.564009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fcb272-6d82-491c-b1dc-2e7b61073ab7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:54.565973 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fcb272-6d82-491c-b1dc-2e7b61073ab7 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b5fcb272-6d82-491c-b1dc-2e7b61073ab7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:57:54.579204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:54.580188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:54.581221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:54.581221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:54.581221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:57:54.581221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:57:54.581221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:57:54.581221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:57:54.581221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:57:54.581221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:57:54.581221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:57:54.581221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:54.584905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 280/564] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Aug 24 09:57:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:54.585971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcb272-6d82-491c-b1dc-2e7b61073ab7 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:54.637888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef91c20-b046-4650-b47c-29de982a8f7a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:54.691499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f825aa-c928-49af-921a-26ae8486ae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:54.692134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f825aa-c928-49af-921a-26ae8486ae72 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.693139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f825aa-c928-49af-921a-26ae8486ae72 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.693621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f825aa-c928-49af-921a-26ae8486ae72 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.694429 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f825aa-c928-49af-921a-26ae8486ae72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:54.694739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f825aa-c928-49af-921a-26ae8486ae72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:54.696127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f825aa-c928-49af-921a-26ae8486ae72 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:54.712902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 285/565] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:54.712902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f825aa-c928-49af-921a-26ae8486ae72 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:54.721708 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcb272-6d82-491c-b1dc-2e7b61073ab7 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:54.765432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d468882-313b-4695-86b4-df9b649fa421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:54.766260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d468882-313b-4695-86b4-df9b649fa421 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.766699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d468882-313b-4695-86b4-df9b649fa421 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.767165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d468882-313b-4695-86b4-df9b649fa421 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.784122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d468882-313b-4695-86b4-df9b649fa421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-0d468882-313b-4695-86b4-df9b649fa421 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:54.792904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:54.794626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:54.796260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 281/566] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:54.797610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d468882-313b-4695-86b4-df9b649fa421 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:54.798884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:54.827418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:54.837916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f825aa-c928-49af-921a-26ae8486ae72 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:54.875433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739e8af-1ea0-4935-ba02-eebc0cc8eabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:54.876039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739e8af-1ea0-4935-ba02-eebc0cc8eabc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.876343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739e8af-1ea0-4935-ba02-eebc0cc8eabc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.876647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739e8af-1ea0-4935-ba02-eebc0cc8eabc None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.877415 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739e8af-1ea0-4935-ba02-eebc0cc8eabc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:54.877721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739e8af-1ea0-4935-ba02-eebc0cc8eabc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:54.879168 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739e8af-1ea0-4935-ba02-eebc0cc8eabc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:54.897481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 286/567] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Aug 24 09:57:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:54.898338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739e8af-1ea0-4935-ba02-eebc0cc8eabc None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:54.905010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d468882-313b-4695-86b4-df9b649fa421 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:54.943116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e914cf-5226-491c-a490-4ed7e1039aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:54.943804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e914cf-5226-491c-a490-4ed7e1039aef None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:54.944090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e914cf-5226-491c-a490-4ed7e1039aef None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:54.944361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e914cf-5226-491c-a490-4ed7e1039aef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:54.954548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e914cf-5226-491c-a490-4ed7e1039aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-90e914cf-5226-491c-a490-4ed7e1039aef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:54.960302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:54.961472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:54.962324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 282/568] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:54.963156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e914cf-5226-491c-a490-4ed7e1039aef None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.010787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739e8af-1ea0-4935-ba02-eebc0cc8eabc None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.038353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:55.046197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba203a-cfc0-43a3-99e2-d01375e1d6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.046825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba203a-cfc0-43a3-99e2-d01375e1d6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.049722 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba203a-cfc0-43a3-99e2-d01375e1d6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.049722 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba203a-cfc0-43a3-99e2-d01375e1d6c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.052472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:55.055502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba203a-cfc0-43a3-99e2-d01375e1d6c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:55.056973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba203a-cfc0-43a3-99e2-d01375e1d6c8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:55.067653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e914cf-5226-491c-a490-4ed7e1039aef None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-51ba203a-cfc0-43a3-99e2-d01375e1d6c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:57:55.069775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:55.074987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:55.076378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:57:55.076378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:57:55.076378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:57:55.076378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:57:55.076378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:57:55.076378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:57:55.076378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:55.076378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:55.076378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 287/569] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:55.076378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba203a-cfc0-43a3-99e2-d01375e1d6c8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.107733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9d308-a429-4a8d-bb0f-4de1a4860925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.108364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9d308-a429-4a8d-bb0f-4de1a4860925 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.108638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9d308-a429-4a8d-bb0f-4de1a4860925 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.108934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9d308-a429-4a8d-bb0f-4de1a4860925 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.120214 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9d308-a429-4a8d-bb0f-4de1a4860925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-23a9d308-a429-4a8d-bb0f-4de1a4860925 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:55.125587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:55.126676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 283/570] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:55.129782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9d308-a429-4a8d-bb0f-4de1a4860925 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.187745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba203a-cfc0-43a3-99e2-d01375e1d6c8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.225491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e4ef0-62e2-47e8-8daa-c14b3d310ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.226214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e4ef0-62e2-47e8-8daa-c14b3d310ffb None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.226561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e4ef0-62e2-47e8-8daa-c14b3d310ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.226899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e4ef0-62e2-47e8-8daa-c14b3d310ffb None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.227633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e4ef0-62e2-47e8-8daa-c14b3d310ffb None admin] RBAC: Adding query filter params (name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:55.228130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e4ef0-62e2-47e8-8daa-c14b3d310ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:55.229347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e4ef0-62e2-47e8-8daa-c14b3d310ffb None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:55.233931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9d308-a429-4a8d-bb0f-4de1a4860925 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.237882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 288/571] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:55.238766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e4ef0-62e2-47e8-8daa-c14b3d310ffb None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.283139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d69694-a794-49aa-a0a3-692f9a67a7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.283796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d69694-a794-49aa-a0a3-692f9a67a7bf None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.284109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d69694-a794-49aa-a0a3-692f9a67a7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.285478 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d69694-a794-49aa-a0a3-692f9a67a7bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.296919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d69694-a794-49aa-a0a3-692f9a67a7bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:55.298525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d69694-a794-49aa-a0a3-692f9a67a7bf None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:55.306756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-72d69694-a794-49aa-a0a3-692f9a67a7bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:55.314602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:55.315750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:55.316672 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:55.316672 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:55.316672 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:55.316852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:55.316967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 284/572] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:55.318499 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d69694-a794-49aa-a0a3-692f9a67a7bf None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.378231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e4ef0-62e2-47e8-8daa-c14b3d310ffb None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.417635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6954f-09f1-409b-af26-40c9a1783aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.418248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6954f-09f1-409b-af26-40c9a1783aba None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.418531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6954f-09f1-409b-af26-40c9a1783aba None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.418803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6954f-09f1-409b-af26-40c9a1783aba None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.419464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6954f-09f1-409b-af26-40c9a1783aba None admin] RBAC: Adding query filter params (name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:55.419764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6954f-09f1-409b-af26-40c9a1783aba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:55.421158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6954f-09f1-409b-af26-40c9a1783aba None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:55.430755 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 289/573] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:55.431684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6954f-09f1-409b-af26-40c9a1783aba None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.438732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d69694-a794-49aa-a0a3-692f9a67a7bf None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.474190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc5b6a-bca4-40ed-a070-23d97ec09706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.474945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc5b6a-bca4-40ed-a070-23d97ec09706 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.475222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc5b6a-bca4-40ed-a070-23d97ec09706 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.475500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc5b6a-bca4-40ed-a070-23d97ec09706 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.476594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc5b6a-bca4-40ed-a070-23d97ec09706 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:55.476594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc5b6a-bca4-40ed-a070-23d97ec09706 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:55.478050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc5b6a-bca4-40ed-a070-23d97ec09706 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:55.491466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 285/574] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:55.492401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc5b6a-bca4-40ed-a070-23d97ec09706 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.536191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6954f-09f1-409b-af26-40c9a1783aba None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.550160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:55.563479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:55.575325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953d8a5-5605-4fc6-8f1d-1aa41f02de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.575930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d8a5-5605-4fc6-8f1d-1aa41f02de4c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.576234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d8a5-5605-4fc6-8f1d-1aa41f02de4c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.576510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d8a5-5605-4fc6-8f1d-1aa41f02de4c None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.577211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953d8a5-5605-4fc6-8f1d-1aa41f02de4c None admin] RBAC: Adding query filter params (name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:55.577514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953d8a5-5605-4fc6-8f1d-1aa41f02de4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:55.578888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953d8a5-5605-4fc6-8f1d-1aa41f02de4c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:55.591360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 290/575] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:55.592201 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953d8a5-5605-4fc6-8f1d-1aa41f02de4c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.608030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc5b6a-bca4-40ed-a070-23d97ec09706 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.645369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3329b74-0e1c-40d6-8e44-f79cbf0db791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.645943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3329b74-0e1c-40d6-8e44-f79cbf0db791 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.646232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3329b74-0e1c-40d6-8e44-f79cbf0db791 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.646504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3329b74-0e1c-40d6-8e44-f79cbf0db791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.657110 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3329b74-0e1c-40d6-8e44-f79cbf0db791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:55.658569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3329b74-0e1c-40d6-8e44-f79cbf0db791 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-c3329b74-0e1c-40d6-8e44-f79cbf0db791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:55.674623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:55.681456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:55.682656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:55.682656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:55.682656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:55.682656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 286/576] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:55.682656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3329b74-0e1c-40d6-8e44-f79cbf0db791 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.713051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953d8a5-5605-4fc6-8f1d-1aa41f02de4c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.755189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa99b3c-905c-4e6d-aecc-79428e8d76d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.755790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa99b3c-905c-4e6d-aecc-79428e8d76d1 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.756217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa99b3c-905c-4e6d-aecc-79428e8d76d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.756346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa99b3c-905c-4e6d-aecc-79428e8d76d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.757799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa99b3c-905c-4e6d-aecc-79428e8d76d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:55.757799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa99b3c-905c-4e6d-aecc-79428e8d76d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:55.758671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa99b3c-905c-4e6d-aecc-79428e8d76d1 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:55.771845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 291/577] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:55.772686 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa99b3c-905c-4e6d-aecc-79428e8d76d1 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.784750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3329b74-0e1c-40d6-8e44-f79cbf0db791 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.796517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:55.815743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:55.822793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1d1e0-93a9-42b6-9219-99c12f262f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.823434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1d1e0-93a9-42b6-9219-99c12f262f7c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.823720 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1d1e0-93a9-42b6-9219-99c12f262f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.823997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1d1e0-93a9-42b6-9219-99c12f262f7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.838849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1d1e0-93a9-42b6-9219-99c12f262f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:55.843955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1d1e0-93a9-42b6-9219-99c12f262f7c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-e2f1d1e0-93a9-42b6-9219-99c12f262f7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:55.856207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:55.857250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:55.858163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:55.858163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:55.858163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:55.858163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 287/578] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:55.858163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1d1e0-93a9-42b6-9219-99c12f262f7c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.897136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa99b3c-905c-4e6d-aecc-79428e8d76d1 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.934535 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985bf628-197a-4eb2-8c0c-616b1ef22947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.935126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bf628-197a-4eb2-8c0c-616b1ef22947 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.935401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bf628-197a-4eb2-8c0c-616b1ef22947 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.935670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bf628-197a-4eb2-8c0c-616b1ef22947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.936390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985bf628-197a-4eb2-8c0c-616b1ef22947 None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=88fe81f7b4c04e9f9a26b430b39ccd36) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:55.936702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985bf628-197a-4eb2-8c0c-616b1ef22947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:55.938155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985bf628-197a-4eb2-8c0c-616b1ef22947 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:55.943776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 292/579] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/role_assignments?user.id=88fe81f7b4c04e9f9a26b430b39ccd36&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:55.944598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985bf628-197a-4eb2-8c0c-616b1ef22947 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.972440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1d1e0-93a9-42b6-9219-99c12f262f7c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:56.021073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804eb5ee-ac2b-4e17-96d9-e28de068bcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:56.021722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eb5ee-ac2b-4e17-96d9-e28de068bcbf None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.021997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eb5ee-ac2b-4e17-96d9-e28de068bcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.022269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eb5ee-ac2b-4e17-96d9-e28de068bcbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.034479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804eb5ee-ac2b-4e17-96d9-e28de068bcbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:56.036566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804eb5ee-ac2b-4e17-96d9-e28de068bcbf None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-804eb5ee-ac2b-4e17-96d9-e28de068bcbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:56.055107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:56.057870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:56.060276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:56.060276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:56.060276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:56.060276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 288/580] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:56.060276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804eb5ee-ac2b-4e17-96d9-e28de068bcbf None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:56.077433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:56.090040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:56.094188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985bf628-197a-4eb2-8c0c-616b1ef22947 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:56.135993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45de82-ce68-4dad-977a-51de9c0236de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:56.136636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45de82-ce68-4dad-977a-51de9c0236de None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.136913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45de82-ce68-4dad-977a-51de9c0236de None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.137191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45de82-ce68-4dad-977a-51de9c0236de None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.137844 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45de82-ce68-4dad-977a-51de9c0236de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:56.138150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45de82-ce68-4dad-977a-51de9c0236de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:56.139554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45de82-ce68-4dad-977a-51de9c0236de None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:56.158251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 293/581] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:56.158251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45de82-ce68-4dad-977a-51de9c0236de None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:56.180953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804eb5ee-ac2b-4e17-96d9-e28de068bcbf None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:56.226794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791063ad-f2d9-4e92-98c7-0c09581f4fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:56.227472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791063ad-f2d9-4e92-98c7-0c09581f4fae None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.227765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791063ad-f2d9-4e92-98c7-0c09581f4fae None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.228024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791063ad-f2d9-4e92-98c7-0c09581f4fae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.235264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791063ad-f2d9-4e92-98c7-0c09581f4fae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:56.236765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791063ad-f2d9-4e92-98c7-0c09581f4fae None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-791063ad-f2d9-4e92-98c7-0c09581f4fae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:57:56.248824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:56.250022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:56.251245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:57:56.251245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:57:56.251245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:57:56.251245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:57:56.251245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:57:56.251245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:57:56.251245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:56.251245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:56.255221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 289/582] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:56.255221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791063ad-f2d9-4e92-98c7-0c09581f4fae None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:56.290775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45de82-ce68-4dad-977a-51de9c0236de None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:56.333467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0e255-b403-4286-b073-3f9a6dce11c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:56.334393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e255-b403-4286-b073-3f9a6dce11c4 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.334845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e255-b403-4286-b073-3f9a6dce11c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.335286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e255-b403-4286-b073-3f9a6dce11c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.335975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0e255-b403-4286-b073-3f9a6dce11c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:56.336306 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0e255-b403-4286-b073-3f9a6dce11c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:56.337766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0e255-b403-4286-b073-3f9a6dce11c4 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:56.351188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 294/583] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:56.351788 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:56.352131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0e255-b403-4286-b073-3f9a6dce11c4 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:56.372995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:56.382563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791063ad-f2d9-4e92-98c7-0c09581f4fae None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:56.426113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5036490-765b-4122-a5fa-967478b94fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:56.426958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5036490-765b-4122-a5fa-967478b94fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.427415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5036490-765b-4122-a5fa-967478b94fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.427797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5036490-765b-4122-a5fa-967478b94fc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.428685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5036490-765b-4122-a5fa-967478b94fc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:56.429021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5036490-765b-4122-a5fa-967478b94fc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:56.430387 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5036490-765b-4122-a5fa-967478b94fc6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:56.444671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 290/584] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:56.445926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5036490-765b-4122-a5fa-967478b94fc6 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:56.483502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0e255-b403-4286-b073-3f9a6dce11c4 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:56.523466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7e098-34fa-4a89-b07f-300bfae1cac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:56.524107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7e098-34fa-4a89-b07f-300bfae1cac7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.524499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7e098-34fa-4a89-b07f-300bfae1cac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.525005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7e098-34fa-4a89-b07f-300bfae1cac7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.534925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7e098-34fa-4a89-b07f-300bfae1cac7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:56.536439 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7e098-34fa-4a89-b07f-300bfae1cac7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-c0b7e098-34fa-4a89-b07f-300bfae1cac7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:57:56.550284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:56.551528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:56.552480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:56.552480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:56.552480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:57:56.552480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:57:56.552480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:57:56.552480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:57:56.552480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:57:56.552480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:57:56.552480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:57:56.552480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:56.556608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 295/585] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:56.557604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7e098-34fa-4a89-b07f-300bfae1cac7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:56.566664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5036490-765b-4122-a5fa-967478b94fc6 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:56.606322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae229df-0e20-466a-a00c-7ad5a022a66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:56.606933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae229df-0e20-466a-a00c-7ad5a022a66d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.607210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae229df-0e20-466a-a00c-7ad5a022a66d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.607479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae229df-0e20-466a-a00c-7ad5a022a66d None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.608268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae229df-0e20-466a-a00c-7ad5a022a66d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:56.608463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae229df-0e20-466a-a00c-7ad5a022a66d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:56.610017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae229df-0e20-466a-a00c-7ad5a022a66d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:56.611897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:56.623172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 291/586] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:57:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:56.624273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae229df-0e20-466a-a00c-7ad5a022a66d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:56.657962 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:56.684151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7e098-34fa-4a89-b07f-300bfae1cac7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:56.727727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043bc42-c32e-4c4c-8571-92fac9c4b6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:56.729103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043bc42-c32e-4c4c-8571-92fac9c4b6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.729383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043bc42-c32e-4c4c-8571-92fac9c4b6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.729663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043bc42-c32e-4c4c-8571-92fac9c4b6b4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.737459 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043bc42-c32e-4c4c-8571-92fac9c4b6b4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:56.738929 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043bc42-c32e-4c4c-8571-92fac9c4b6b4 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-b043bc42-c32e-4c4c-8571-92fac9c4b6b4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:57:56.750076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:56.751080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 296/587] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:57:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:56.752045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043bc42-c32e-4c4c-8571-92fac9c4b6b4 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:56.763829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae229df-0e20-466a-a00c-7ad5a022a66d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:56.806263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bed244-4814-407f-8b0b-7f2e871b6db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:56.807205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bed244-4814-407f-8b0b-7f2e871b6db5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.807685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bed244-4814-407f-8b0b-7f2e871b6db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.808266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bed244-4814-407f-8b0b-7f2e871b6db5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.820167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bed244-4814-407f-8b0b-7f2e871b6db5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:56.824381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bed244-4814-407f-8b0b-7f2e871b6db5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-f0bed244-4814-407f-8b0b-7f2e871b6db5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:57:56.837624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:56.838648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 292/588] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:56.839640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bed244-4814-407f-8b0b-7f2e871b6db5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:56.878229 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043bc42-c32e-4c4c-8571-92fac9c4b6b4 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:56.902311 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:56.916600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3ca3-0a59-40e4-a518-1aae1f3ccea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:56.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3ca3-0a59-40e4-a518-1aae1f3ccea7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3ca3-0a59-40e4-a518-1aae1f3ccea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3ca3-0a59-40e4-a518-1aae1f3ccea7 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff3ca3-0a59-40e4-a518-1aae1f3ccea7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:56.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff3ca3-0a59-40e4-a518-1aae1f3ccea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:56.920415 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff3ca3-0a59-40e4-a518-1aae1f3ccea7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:56.930488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:56.948556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 297/589] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Aug 24 09:57:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:56.948556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3ca3-0a59-40e4-a518-1aae1f3ccea7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:56.967611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bed244-4814-407f-8b0b-7f2e871b6db5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:57.008753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b6c9f-4f8d-490a-8d43-167e74b662ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:57.009655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b6c9f-4f8d-490a-8d43-167e74b662ea None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.010466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b6c9f-4f8d-490a-8d43-167e74b662ea None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.010945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b6c9f-4f8d-490a-8d43-167e74b662ea None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/7cd6d2291f934fbdb25c410c2c3abcb6/roles/f9446582c6a34010bd3ae841eba530bc` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.027246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b6c9f-4f8d-490a-8d43-167e74b662ea None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=7cd6d2291f934fbdb25c410c2c3abcb6, role_id=f9446582c6a34010bd3ae841eba530bc)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:57.028739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b6c9f-4f8d-490a-8d43-167e74b662ea None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:57.063149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3ca3-0a59-40e4-a518-1aae1f3ccea7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:57.103155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb65c2e-2a8b-4fd9-82db-9a62b3665be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:57.103767 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb65c2e-2a8b-4fd9-82db-9a62b3665be2 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.104047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb65c2e-2a8b-4fd9-82db-9a62b3665be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.104320 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb65c2e-2a8b-4fd9-82db-9a62b3665be2 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.105079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb65c2e-2a8b-4fd9-82db-9a62b3665be2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:57.105383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb65c2e-2a8b-4fd9-82db-9a62b3665be2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:57.107109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb65c2e-2a8b-4fd9-82db-9a62b3665be2 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:57.125061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 293/590] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:57:56 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/7cd6d2291f934fbdb25c410c2c3abcb6/roles/f9446582c6a34010bd3ae841eba530bc => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:57.125061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b6c9f-4f8d-490a-8d43-167e74b662ea None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:57.127966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 298/591] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Aug 24 09:57:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:57.128852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb65c2e-2a8b-4fd9-82db-9a62b3665be2 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:57.199586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:57.255384 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb65c2e-2a8b-4fd9-82db-9a62b3665be2 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:57.297698 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b6c9f-4f8d-490a-8d43-167e74b662ea None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:57.303174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:57.313090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38618b-3f6e-4cd9-920a-b4e1c622461e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:57.313741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38618b-3f6e-4cd9-920a-b4e1c622461e None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.314076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38618b-3f6e-4cd9-920a-b4e1c622461e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.314367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38618b-3f6e-4cd9-920a-b4e1c622461e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.326837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38618b-3f6e-4cd9-920a-b4e1c622461e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:57.328444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38618b-3f6e-4cd9-920a-b4e1c622461e None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:57.345714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05dcf1-e45f-42ac-9f5a-ca593bfa6f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-ce38618b-3f6e-4cd9-920a-b4e1c622461e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:57.346330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:57.347675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:57.349687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:57.349687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:57.349687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:57.349870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05dcf1-e45f-42ac-9f5a-ca593bfa6f04 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.349870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05dcf1-e45f-42ac-9f5a-ca593bfa6f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.349870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05dcf1-e45f-42ac-9f5a-ca593bfa6f04 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.349870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05dcf1-e45f-42ac-9f5a-ca593bfa6f04 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:57.350111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 299/592] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:57.350177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05dcf1-e45f-42ac-9f5a-ca593bfa6f04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:57.350244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38618b-3f6e-4cd9-920a-b4e1c622461e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:57.351336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05dcf1-e45f-42ac-9f5a-ca593bfa6f04 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:57.381449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 294/593] 10.222.0.30 () {56 vars in 1139 bytes} [Wed Aug 24 09:57:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:57.382326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05dcf1-e45f-42ac-9f5a-ca593bfa6f04 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:57.507721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05dcf1-e45f-42ac-9f5a-ca593bfa6f04 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:57.551048 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39640cb4-f1f1-408e-b64c-3292bb1e5df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:57.552207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39640cb4-f1f1-408e-b64c-3292bb1e5df9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.552483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39640cb4-f1f1-408e-b64c-3292bb1e5df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.552753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39640cb4-f1f1-408e-b64c-3292bb1e5df9 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.553570 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39640cb4-f1f1-408e-b64c-3292bb1e5df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:57.553874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39640cb4-f1f1-408e-b64c-3292bb1e5df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:57.555261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38618b-3f6e-4cd9-920a-b4e1c622461e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:57.555460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39640cb4-f1f1-408e-b64c-3292bb1e5df9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:57.573922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 295/594] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:57.574652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:57.574991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39640cb4-f1f1-408e-b64c-3292bb1e5df9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:57.605212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:57.611506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470016c0-841f-4b1c-8161-aa847a78bfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:57.612143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470016c0-841f-4b1c-8161-aa847a78bfdc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.612423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470016c0-841f-4b1c-8161-aa847a78bfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.612695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470016c0-841f-4b1c-8161-aa847a78bfdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.625660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470016c0-841f-4b1c-8161-aa847a78bfdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:57.627214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470016c0-841f-4b1c-8161-aa847a78bfdc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-470016c0-841f-4b1c-8161-aa847a78bfdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:57.644445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:57.645525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:57.646432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:57.646432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:57.646432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:57.646432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 300/595] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:57.647087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470016c0-841f-4b1c-8161-aa847a78bfdc None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:57.718655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39640cb4-f1f1-408e-b64c-3292bb1e5df9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:57.763037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3c5dc-9997-41a8-a5ef-2fb9cf38812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:57.763579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3c5dc-9997-41a8-a5ef-2fb9cf38812b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.763868 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3c5dc-9997-41a8-a5ef-2fb9cf38812b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.764208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3c5dc-9997-41a8-a5ef-2fb9cf38812b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.775015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470016c0-841f-4b1c-8161-aa847a78bfdc None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:57.779665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3c5dc-9997-41a8-a5ef-2fb9cf38812b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:57.780747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3c5dc-9997-41a8-a5ef-2fb9cf38812b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-a5d3c5dc-9997-41a8-a5ef-2fb9cf38812b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:57.798248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:57.799264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:57.800147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:57.800147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:57.800147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:57.800147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 296/596] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:57.800809 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3c5dc-9997-41a8-a5ef-2fb9cf38812b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:57.825103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd07987-3157-4768-abc0-5c83155bdb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:57.826606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd07987-3157-4768-abc0-5c83155bdb2c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.827060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd07987-3157-4768-abc0-5c83155bdb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.827790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd07987-3157-4768-abc0-5c83155bdb2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.839039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd07987-3157-4768-abc0-5c83155bdb2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:57.841010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd07987-3157-4768-abc0-5c83155bdb2c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-fbd07987-3157-4768-abc0-5c83155bdb2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:57:57.857419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:57.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:57.859666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:57:57.859666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:57:57.859666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:57:57.859666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:57:57.859666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:57:57.859666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:57:57.859666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:57.859666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:57.862405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 301/597] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:57.863513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd07987-3157-4768-abc0-5c83155bdb2c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:57.891438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:57.894934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:57.924302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3c5dc-9997-41a8-a5ef-2fb9cf38812b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:57.971552 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b7302-e368-498e-8efc-95954ad743c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:57.972767 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b7302-e368-498e-8efc-95954ad743c4 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.974558 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b7302-e368-498e-8efc-95954ad743c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.975436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b7302-e368-498e-8efc-95954ad743c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.976531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b7302-e368-498e-8efc-95954ad743c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:57.981171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b7302-e368-498e-8efc-95954ad743c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:57.982678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b7302-e368-498e-8efc-95954ad743c4 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:57.997624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 297/598] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:57.998299 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b7302-e368-498e-8efc-95954ad743c4 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:58.010506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd07987-3157-4768-abc0-5c83155bdb2c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:58.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ec27e-f056-415c-9983-683a39731c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:58.049758 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ec27e-f056-415c-9983-683a39731c8b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:58.050133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ec27e-f056-415c-9983-683a39731c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:58.050326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ec27e-f056-415c-9983-683a39731c8b None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:58.051026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ec27e-f056-415c-9983-683a39731c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:58.051359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ec27e-f056-415c-9983-683a39731c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:58.052827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ec27e-f056-415c-9983-683a39731c8b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:58.065981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 302/599] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:58.067703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5d158-89b3-432e-8cf2-b0d53a23babe None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:58.067991 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5d158-89b3-432e-8cf2-b0d53a23babe None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:58.068265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5d158-89b3-432e-8cf2-b0d53a23babe None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:58.072920 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 303/600] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:58.072920 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5d158-89b3-432e-8cf2-b0d53a23babe None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:58.128204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b7302-e368-498e-8efc-95954ad743c4 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:58.163110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:58.168494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1f46c-a915-45f4-aa6c-d0269c7200bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:58.169326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f46c-a915-45f4-aa6c-d0269c7200bf None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:58.169622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f46c-a915-45f4-aa6c-d0269c7200bf None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:58.169893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f46c-a915-45f4-aa6c-d0269c7200bf None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:58.170595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1f46c-a915-45f4-aa6c-d0269c7200bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:58.170904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1f46c-a915-45f4-aa6c-d0269c7200bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:58.172326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1f46c-a915-45f4-aa6c-d0269c7200bf None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:58.174141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:58.189024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 298/601] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:57:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:58.191138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5d158-89b3-432e-8cf2-b0d53a23babe None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:58.194514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1f46c-a915-45f4-aa6c-d0269c7200bf None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:58.231059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46250d44-76fc-4f58-a498-e98d4911b747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:58.231711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46250d44-76fc-4f58-a498-e98d4911b747 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:58.231995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46250d44-76fc-4f58-a498-e98d4911b747 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:58.232264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46250d44-76fc-4f58-a498-e98d4911b747 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:58.240323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46250d44-76fc-4f58-a498-e98d4911b747 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:58.241936 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46250d44-76fc-4f58-a498-e98d4911b747 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-46250d44-76fc-4f58-a498-e98d4911b747 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:57:58.254637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:58.255619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:58.256511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:57:58.256511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:57:58.256511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:57:58.256511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:57:58.256511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:57:58.256511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:57:58.256511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:58.256511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:58.256511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 304/602] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:58.257513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf4aa3-8845-4cf4-803a-ab10d193e04d None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:58.257629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf4aa3-8845-4cf4-803a-ab10d193e04d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:58.257983 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf4aa3-8845-4cf4-803a-ab10d193e04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:58.281041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-3dbf4aa3-8845-4cf4-803a-ab10d193e04d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:58.282267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3dbf4aa3-8845-4cf4-803a-ab10d193e04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:58.328676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1f46c-a915-45f4-aa6c-d0269c7200bf None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:58.369522 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 305/603] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:58.370739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf4aa3-8845-4cf4-803a-ab10d193e04d None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:58.378584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae001cf-a224-4e23-a7c4-4cf0012d6c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:58.380050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae001cf-a224-4e23-a7c4-4cf0012d6c21 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:58.380537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae001cf-a224-4e23-a7c4-4cf0012d6c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:58.381046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae001cf-a224-4e23-a7c4-4cf0012d6c21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:58.390424 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae001cf-a224-4e23-a7c4-4cf0012d6c21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:58.392614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae001cf-a224-4e23-a7c4-4cf0012d6c21 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-eae001cf-a224-4e23-a7c4-4cf0012d6c21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:57:58.407999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:58.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:58.409925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:57:58.409925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:57:58.409925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:57:58.409925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:57:58.409925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:57:58.409925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:57:58.409925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:57:58.409925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:58.417101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 299/604] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:57:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:58.417101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae001cf-a224-4e23-a7c4-4cf0012d6c21 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:58.463800 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:58.477348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:58.505668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf4aa3-8845-4cf4-803a-ab10d193e04d None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:58.529041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae001cf-a224-4e23-a7c4-4cf0012d6c21 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:58.545085 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5dbdf-9555-4b50-ba4d-d181f286dda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:58.546356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5dbdf-9555-4b50-ba4d-d181f286dda9 None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:58.547223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5dbdf-9555-4b50-ba4d-d181f286dda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:58.547876 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5dbdf-9555-4b50-ba4d-d181f286dda9 None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/d0b124ca267b4a4391557ad1396524cf/roles/f9446582c6a34010bd3ae841eba530bc` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:58.566186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a48f10-7ed8-43ae-ae70-94b88d255fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:58.566798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a48f10-7ed8-43ae-ae70-94b88d255fba None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:58.567185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a48f10-7ed8-43ae-ae70-94b88d255fba None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:58.567736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a48f10-7ed8-43ae-ae70-94b88d255fba None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:58.568550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a48f10-7ed8-43ae-ae70-94b88d255fba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:58.568883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a48f10-7ed8-43ae-ae70-94b88d255fba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:58.569484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5dbdf-9555-4b50-ba4d-d181f286dda9 None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=d0b124ca267b4a4391557ad1396524cf, role_id=f9446582c6a34010bd3ae841eba530bc)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:58.570320 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a48f10-7ed8-43ae-ae70-94b88d255fba None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:58.571320 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5dbdf-9555-4b50-ba4d-d181f286dda9 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:58.592185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 300/605] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:57:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:58.592185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a48f10-7ed8-43ae-ae70-94b88d255fba None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:58.688898 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 306/606] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:57:58 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/d0b124ca267b4a4391557ad1396524cf/roles/f9446582c6a34010bd3ae841eba530bc => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:58.688898 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5dbdf-9555-4b50-ba4d-d181f286dda9 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:58.726152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a48f10-7ed8-43ae-ae70-94b88d255fba None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:58.749189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:58.785878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128f753-a0f3-4625-95ed-f1bb7c738479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:58.786760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f753-a0f3-4625-95ed-f1bb7c738479 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:58.787251 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f753-a0f3-4625-95ed-f1bb7c738479 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:58.787996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f753-a0f3-4625-95ed-f1bb7c738479 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:58.789502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128f753-a0f3-4625-95ed-f1bb7c738479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:58.789820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128f753-a0f3-4625-95ed-f1bb7c738479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:58.791619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128f753-a0f3-4625-95ed-f1bb7c738479 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:58.811255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 301/607] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:57:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:58.812281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128f753-a0f3-4625-95ed-f1bb7c738479 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:58.825611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5dbdf-9555-4b50-ba4d-d181f286dda9 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:58.849714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:58.892173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14927f9b-8ad1-46e0-ade4-fdc48e3eb806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:58.892953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927f9b-8ad1-46e0-ade4-fdc48e3eb806 None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:58.893257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927f9b-8ad1-46e0-ade4-fdc48e3eb806 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:58.893518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927f9b-8ad1-46e0-ade4-fdc48e3eb806 None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/c71783d1327a47149a74414218dd7219/roles/f9446582c6a34010bd3ae841eba530bc` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:58.915305 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14927f9b-8ad1-46e0-ade4-fdc48e3eb806 None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=c71783d1327a47149a74414218dd7219, role_id=f9446582c6a34010bd3ae841eba530bc)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:58.916851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14927f9b-8ad1-46e0-ade4-fdc48e3eb806 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:58.950562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128f753-a0f3-4625-95ed-f1bb7c738479 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:58.991315 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f0c1-6011-40b9-8b94-be83a64e0668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:58.992371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f0c1-6011-40b9-8b94-be83a64e0668 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:58.992840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f0c1-6011-40b9-8b94-be83a64e0668 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:58.993355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f0c1-6011-40b9-8b94-be83a64e0668 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:59.006123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6f0c1-6011-40b9-8b94-be83a64e0668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-70a6f0c1-6011-40b9-8b94-be83a64e0668 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:57:59.012090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:59.013099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:57:59.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:59.015959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 302/608] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:57:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:59.017676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f0c1-6011-40b9-8b94-be83a64e0668 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:59.041687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 307/609] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:57:58 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/c71783d1327a47149a74414218dd7219/roles/f9446582c6a34010bd3ae841eba530bc => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:59.042951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14927f9b-8ad1-46e0-ade4-fdc48e3eb806 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:59.119789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:59.229786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14927f9b-8ad1-46e0-ade4-fdc48e3eb806 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:59.256551 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f0c1-6011-40b9-8b94-be83a64e0668 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:59.283393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc7d7f-f923-4c8c-a1bf-f3d75cf99c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:59.284228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc7d7f-f923-4c8c-a1bf-f3d75cf99c91 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:59.284519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc7d7f-f923-4c8c-a1bf-f3d75cf99c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:59.284797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc7d7f-f923-4c8c-a1bf-f3d75cf99c91 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:59.285526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc7d7f-f923-4c8c-a1bf-f3d75cf99c91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:59.285834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc7d7f-f923-4c8c-a1bf-f3d75cf99c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:59.287540 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc7d7f-f923-4c8c-a1bf-f3d75cf99c91 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:59.303514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 308/610] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:57:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:59.305102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c18f7f-693b-43f7-972a-c943019cf30d None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:59.305561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c18f7f-693b-43f7-972a-c943019cf30d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:59.305914 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c18f7f-693b-43f7-972a-c943019cf30d None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:59.312009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 309/611] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:59.312009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c18f7f-693b-43f7-972a-c943019cf30d None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:59.325060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded4db2-2719-4046-9d13-4a1339c31384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:59.326660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4db2-2719-4046-9d13-4a1339c31384 None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:59.327525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4db2-2719-4046-9d13-4a1339c31384 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:59.327986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4db2-2719-4046-9d13-4a1339c31384 None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/ae9b6525009b41288dd918437acee92a/roles/f9446582c6a34010bd3ae841eba530bc` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:59.349927 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:59.353500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded4db2-2719-4046-9d13-4a1339c31384 None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=ae9b6525009b41288dd918437acee92a, role_id=f9446582c6a34010bd3ae841eba530bc)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:59.355464 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded4db2-2719-4046-9d13-4a1339c31384 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:59.433668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c18f7f-693b-43f7-972a-c943019cf30d None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:59.469866 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3025565-48b8-45c5-a71a-036137783bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:59.470486 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3025565-48b8-45c5-a71a-036137783bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:59.470768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3025565-48b8-45c5-a71a-036137783bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:59.471039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3025565-48b8-45c5-a71a-036137783bb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:59.482745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3025565-48b8-45c5-a71a-036137783bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:59.484350 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3025565-48b8-45c5-a71a-036137783bb8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:59.497709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 303/612] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:57:59 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/ae9b6525009b41288dd918437acee92a/roles/f9446582c6a34010bd3ae841eba530bc => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:59.499989 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597582a-cb03-499f-80bd-013b5ec6201a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:59.508021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597582a-cb03-499f-80bd-013b5ec6201a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:59.508021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597582a-cb03-499f-80bd-013b5ec6201a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-d3025565-48b8-45c5-a71a-036137783bb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:57:59.508244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:59.509498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:57:59.510407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:57:59.510407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:57:59.510407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:57:59.510407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 310/613] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:57:59.518141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3025565-48b8-45c5-a71a-036137783bb8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:59.546144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-9597582a-cb03-499f-80bd-013b5ec6201a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:59.547914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9597582a-cb03-499f-80bd-013b5ec6201a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:59.681061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 304/614] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:57:59.720024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:59.740476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3025565-48b8-45c5-a71a-036137783bb8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:59.783652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf45c0-71f4-4661-88e8-b43b9fdff487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:59.784262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf45c0-71f4-4661-88e8-b43b9fdff487 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:59.784584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf45c0-71f4-4661-88e8-b43b9fdff487 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:59.785021 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf45c0-71f4-4661-88e8-b43b9fdff487 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:59.785729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf45c0-71f4-4661-88e8-b43b9fdff487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:59.786234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf45c0-71f4-4661-88e8-b43b9fdff487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:59.788139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf45c0-71f4-4661-88e8-b43b9fdff487 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:59.802171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 311/615] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:57:59.807836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597582a-cb03-499f-80bd-013b5ec6201a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:59.836888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:59.932394 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597582a-cb03-499f-80bd-013b5ec6201a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:59.970158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea92ee-fc33-4237-8dc4-e6ab4235720b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:59.971118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea92ee-fc33-4237-8dc4-e6ab4235720b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:59.971619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea92ee-fc33-4237-8dc4-e6ab4235720b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:59.972126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea92ee-fc33-4237-8dc4-e6ab4235720b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:59.980071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea92ee-fc33-4237-8dc4-e6ab4235720b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:59.981951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea92ee-fc33-4237-8dc4-e6ab4235720b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-faea92ee-fc33-4237-8dc4-e6ab4235720b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:57:59.995142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:57:59.996185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:57:59.997181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:57:59.997181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:57:59.997181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:57:59.997181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:57:59.997181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:57:59.997181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:57:59.997181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:57:59.997181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:57:59.997181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:57:59.997181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:57:59.999918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 305/616] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:57:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:00.008060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf45c0-71f4-4661-88e8-b43b9fdff487 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:00.090990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea92ee-fc33-4237-8dc4-e6ab4235720b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:00.181241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:00.206638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea92ee-fc33-4237-8dc4-e6ab4235720b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:00.208560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf45c0-71f4-4661-88e8-b43b9fdff487 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:00.244352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c235a-60bd-4dcd-9add-2b8ad8007379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:00.245405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c235a-60bd-4dcd-9add-2b8ad8007379 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:00.245909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c235a-60bd-4dcd-9add-2b8ad8007379 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:00.246384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c235a-60bd-4dcd-9add-2b8ad8007379 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:00.260145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c235a-60bd-4dcd-9add-2b8ad8007379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:00.261572 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3de98-dc88-49c2-ad80-b7a29d8708d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:00.262195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3de98-dc88-49c2-ad80-b7a29d8708d0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:00.262476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3de98-dc88-49c2-ad80-b7a29d8708d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:00.262750 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3de98-dc88-49c2-ad80-b7a29d8708d0 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:00.263491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3de98-dc88-49c2-ad80-b7a29d8708d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:00.263804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3de98-dc88-49c2-ad80-b7a29d8708d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:00.265321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3de98-dc88-49c2-ad80-b7a29d8708d0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-325c235a-60bd-4dcd-9add-2b8ad8007379 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:00.268710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:00.269728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:00.272286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:00.282496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 306/617] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:58:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:00.285262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c235a-60bd-4dcd-9add-2b8ad8007379 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:00.311022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:00.311495 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 312/618] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Aug 24 09:58:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:00.321381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3de98-dc88-49c2-ad80-b7a29d8708d0 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:00.468047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3de98-dc88-49c2-ad80-b7a29d8708d0 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:00.473934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c235a-60bd-4dcd-9add-2b8ad8007379 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:00.509168 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69d8ca-e055-4f55-bb86-5dddf88fcb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:00.510122 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69d8ca-e055-4f55-bb86-5dddf88fcb3c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:00.510659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69d8ca-e055-4f55-bb86-5dddf88fcb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:00.511208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69d8ca-e055-4f55-bb86-5dddf88fcb3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:00.522797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69d8ca-e055-4f55-bb86-5dddf88fcb3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:00.524686 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69d8ca-e055-4f55-bb86-5dddf88fcb3c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-1d69d8ca-e055-4f55-bb86-5dddf88fcb3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:00.540903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:00.541917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:00.542811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:00.542811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:00.542811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:00.545027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 313/619] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:00.550934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7b695-9bab-4988-9563-503a987c6f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:00.552179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7b695-9bab-4988-9563-503a987c6f5d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:00.552456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7b695-9bab-4988-9563-503a987c6f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:00.552729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7b695-9bab-4988-9563-503a987c6f5d None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:00.553090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69d8ca-e055-4f55-bb86-5dddf88fcb3c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:00.553423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7b695-9bab-4988-9563-503a987c6f5d None admin] RBAC: Adding query filter params (name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:00.553721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7b695-9bab-4988-9563-503a987c6f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:00.555100 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7b695-9bab-4988-9563-503a987c6f5d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:00.556673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:00.573942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 307/620] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:58:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:00.580312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7b695-9bab-4988-9563-503a987c6f5d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:00.617240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:00.663865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69d8ca-e055-4f55-bb86-5dddf88fcb3c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:00.703301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b715ea-77e4-451d-b80b-17999a174321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:00.704286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b715ea-77e4-451d-b80b-17999a174321 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:00.704748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b715ea-77e4-451d-b80b-17999a174321 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:00.705221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b715ea-77e4-451d-b80b-17999a174321 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:00.706131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b715ea-77e4-451d-b80b-17999a174321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:00.706619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b715ea-77e4-451d-b80b-17999a174321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:00.708574 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b715ea-77e4-451d-b80b-17999a174321 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:00.721890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 314/621] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:00.730750 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b715ea-77e4-451d-b80b-17999a174321 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:00.814434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7b695-9bab-4988-9563-503a987c6f5d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:00.855966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b715ea-77e4-451d-b80b-17999a174321 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:00.871239 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:00.894782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039357ec-04a6-4969-98f0-c4f615b5dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:00.896067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039357ec-04a6-4969-98f0-c4f615b5dd2d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:00.896553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039357ec-04a6-4969-98f0-c4f615b5dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:00.897051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039357ec-04a6-4969-98f0-c4f615b5dd2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:00.904478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039357ec-04a6-4969-98f0-c4f615b5dd2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:00.906607 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039357ec-04a6-4969-98f0-c4f615b5dd2d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:00.914165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50cfc0-518b-4fa0-a7ed-08d6d97654d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:00.914806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50cfc0-518b-4fa0-a7ed-08d6d97654d6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:00.915084 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50cfc0-518b-4fa0-a7ed-08d6d97654d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:00.915354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50cfc0-518b-4fa0-a7ed-08d6d97654d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-039357ec-04a6-4969-98f0-c4f615b5dd2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:00.921244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:00.922166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:00.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:00.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:00.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:00.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:00.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:00.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:00.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:00.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:00.925623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 315/622] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:00.932938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039357ec-04a6-4969-98f0-c4f615b5dd2d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:00.948574 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50cfc0-518b-4fa0-a7ed-08d6d97654d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:00.950494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50cfc0-518b-4fa0-a7ed-08d6d97654d6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-3a50cfc0-518b-4fa0-a7ed-08d6d97654d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:00.978834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:00.979842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:00.981381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:00.981381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:00.981381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:00.981381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 308/623] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:00.990494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50cfc0-518b-4fa0-a7ed-08d6d97654d6 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:01.076038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:01.076367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039357ec-04a6-4969-98f0-c4f615b5dd2d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:01.121813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f328fab-c5c8-4ceb-beb7-ed8b152f80f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:01.122425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f328fab-c5c8-4ceb-beb7-ed8b152f80f2 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:01.122709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f328fab-c5c8-4ceb-beb7-ed8b152f80f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:01.122985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f328fab-c5c8-4ceb-beb7-ed8b152f80f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:01.123718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f328fab-c5c8-4ceb-beb7-ed8b152f80f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:01.124025 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f328fab-c5c8-4ceb-beb7-ed8b152f80f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:01.126124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f328fab-c5c8-4ceb-beb7-ed8b152f80f2 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:01.141163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50cfc0-518b-4fa0-a7ed-08d6d97654d6 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:01.144792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 316/624] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:01.146693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd845a0-19d4-4488-a5c1-b522657c2fd4 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:01.147089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd845a0-19d4-4488-a5c1-b522657c2fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:01.147488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd845a0-19d4-4488-a5c1-b522657c2fd4 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:01.153805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 317/625] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:01.154931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd845a0-19d4-4488-a5c1-b522657c2fd4 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:01.189167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4d747-e39c-42ce-a6d6-dc33467a6348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:01.189967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4d747-e39c-42ce-a6d6-dc33467a6348 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:01.190319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4d747-e39c-42ce-a6d6-dc33467a6348 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:01.190692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4d747-e39c-42ce-a6d6-dc33467a6348 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:01.191561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4d747-e39c-42ce-a6d6-dc33467a6348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:01.192010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4d747-e39c-42ce-a6d6-dc33467a6348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:01.193027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:01.195058 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4d747-e39c-42ce-a6d6-dc33467a6348 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:01.208822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 309/626] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:01.210470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e20ed8-3338-4fc2-bd23-c4b4c78ff2d4 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:01.211365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e20ed8-3338-4fc2-bd23-c4b4c78ff2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:01.211365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e20ed8-3338-4fc2-bd23-c4b4c78ff2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:01.230919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-16e20ed8-3338-4fc2-bd23-c4b4c78ff2d4 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:01.233271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-16e20ed8-3338-4fc2-bd23-c4b4c78ff2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:01.275212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd845a0-19d4-4488-a5c1-b522657c2fd4 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:01.318432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c811fa6-738c-40c0-9732-047d2428415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:01.319375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c811fa6-738c-40c0-9732-047d2428415d None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:01.319798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c811fa6-738c-40c0-9732-047d2428415d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:01.320204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c811fa6-738c-40c0-9732-047d2428415d None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/88fe81f7b4c04e9f9a26b430b39ccd36/roles/f9446582c6a34010bd3ae841eba530bc` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:01.339509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 310/627] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:58:01.339509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e20ed8-3338-4fc2-bd23-c4b4c78ff2d4 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:01.339741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c811fa6-738c-40c0-9732-047d2428415d None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=88fe81f7b4c04e9f9a26b430b39ccd36, role_id=f9446582c6a34010bd3ae841eba530bc)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:01.339741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c811fa6-738c-40c0-9732-047d2428415d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:01.379070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:01.441071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e20ed8-3338-4fc2-bd23-c4b4c78ff2d4 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:01.478269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d696c8-bfee-4820-98b0-f376321b46be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:01.484914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d696c8-bfee-4820-98b0-f376321b46be None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:01.484914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d696c8-bfee-4820-98b0-f376321b46be None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:01.484914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d696c8-bfee-4820-98b0-f376321b46be None admin] PATH_INFO: `/v3/users/nova` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:01.490125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 318/628] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:58:01 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/88fe81f7b4c04e9f9a26b430b39ccd36/roles/f9446582c6a34010bd3ae841eba530bc => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:58:01.496383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d696c8-bfee-4820-98b0-f376321b46be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:01.498299 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d696c8-bfee-4820-98b0-f376321b46be None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-51d696c8-bfee-4820-98b0-f376321b46be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:01.512454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:01.513785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:01.515043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:01.515043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:01.515043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:01.515043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:01.515043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:01.515043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:01.515043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:01.515043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:01.515043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:01.515043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:01.517457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 311/629] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Aug 24 09:58:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:01.527018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c811fa6-738c-40c0-9732-047d2428415d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:01.606251 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8432-35cb-4d51-9a82-43cce4b18a3d None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:01.606859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8432-35cb-4d51-9a82-43cce4b18a3d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:01.607281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8432-35cb-4d51-9a82-43cce4b18a3d None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:01.610079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 312/630] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:01.618586 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52442a1-350c-4664-ac23-0ad81d4195f3 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:01.618968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52442a1-350c-4664-ac23-0ad81d4195f3 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:01.619319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52442a1-350c-4664-ac23-0ad81d4195f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:01.641218 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-d52442a1-350c-4664-ac23-0ad81d4195f3 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:01.642778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d52442a1-350c-4664-ac23-0ad81d4195f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:01.675371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:01.698064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c811fa6-738c-40c0-9732-047d2428415d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:01.729319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 313/631] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:58:01.730639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52442a1-350c-4664-ac23-0ad81d4195f3 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:01.754177 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bb2fb-7913-4963-8d4f-4f800a2ac174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:01.755634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb2fb-7913-4963-8d4f-4f800a2ac174 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:01.756147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb2fb-7913-4963-8d4f-4f800a2ac174 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:01.756660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb2fb-7913-4963-8d4f-4f800a2ac174 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:01.757513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bb2fb-7913-4963-8d4f-4f800a2ac174 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:01.757918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bb2fb-7913-4963-8d4f-4f800a2ac174 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:01.759450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bb2fb-7913-4963-8d4f-4f800a2ac174 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:01.780526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 319/632] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Aug 24 09:58:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:01.780923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:01.786984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bb2fb-7913-4963-8d4f-4f800a2ac174 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:01.886770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52442a1-350c-4664-ac23-0ad81d4195f3 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:01.958743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bb2fb-7913-4963-8d4f-4f800a2ac174 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:01.971887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eee399-032b-4800-a1a4-ae48c410f933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:01.972775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eee399-032b-4800-a1a4-ae48c410f933 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:01.973750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eee399-032b-4800-a1a4-ae48c410f933 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:01.974276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eee399-032b-4800-a1a4-ae48c410f933 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:01.990652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eee399-032b-4800-a1a4-ae48c410f933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-f8eee399-032b-4800-a1a4-ae48c410f933 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:01.995994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:01.997131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:01.997996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:01.998838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 314/633] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:58:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:02.005078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d89ed7-0a25-48c9-8b76-4ecc011de83b None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.005361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d89ed7-0a25-48c9-8b76-4ecc011de83b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.005650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d89ed7-0a25-48c9-8b76-4ecc011de83b None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.007463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 315/634] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:02.008257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d89ed7-0a25-48c9-8b76-4ecc011de83b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:02.014398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7fd2c-2f14-4083-89cb-3c56f961684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:02.015321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7fd2c-2f14-4083-89cb-3c56f961684d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.015824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7fd2c-2f14-4083-89cb-3c56f961684d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.016297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7fd2c-2f14-4083-89cb-3c56f961684d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.031806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:02.095991 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7fd2c-2f14-4083-89cb-3c56f961684d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:02.097517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7fd2c-2f14-4083-89cb-3c56f961684d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:02.120611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:02.135300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d89ed7-0a25-48c9-8b76-4ecc011de83b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:02.182931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ecbcf-459a-4dda-a213-abce89abb7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:02.183629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ecbcf-459a-4dda-a213-abce89abb7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.184053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ecbcf-459a-4dda-a213-abce89abb7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.184336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ecbcf-459a-4dda-a213-abce89abb7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.185157 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ecbcf-459a-4dda-a213-abce89abb7b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:02.185463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ecbcf-459a-4dda-a213-abce89abb7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:02.187101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ecbcf-459a-4dda-a213-abce89abb7b8 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:02.200987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 316/635] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:58:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:02.200987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a44ee-4719-4eb4-b7c5-82a7d27cb2ef None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.200987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a44ee-4719-4eb4-b7c5-82a7d27cb2ef None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.200987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a44ee-4719-4eb4-b7c5-82a7d27cb2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.216428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 320/636] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:58:02.217683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7fd2c-2f14-4083-89cb-3c56f961684d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:02.217794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-706a44ee-4719-4eb4-b7c5-82a7d27cb2ef None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:02.219057 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-706a44ee-4719-4eb4-b7c5-82a7d27cb2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:02.268912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:02.304957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 317/637] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:58:02.305907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a44ee-4719-4eb4-b7c5-82a7d27cb2ef None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:02.335377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7fd2c-2f14-4083-89cb-3c56f961684d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:02.381871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37307df0-a867-4dbb-887a-3d7d4d68dee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:02.382503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37307df0-a867-4dbb-887a-3d7d4d68dee7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.382784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37307df0-a867-4dbb-887a-3d7d4d68dee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.383056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37307df0-a867-4dbb-887a-3d7d4d68dee7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.399470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37307df0-a867-4dbb-887a-3d7d4d68dee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-37307df0-a867-4dbb-887a-3d7d4d68dee7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:02.406454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:02.407488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:02.408406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:02.415996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 321/638] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:58:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:02.415996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37307df0-a867-4dbb-887a-3d7d4d68dee7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:02.432049 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:02.452765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a44ee-4719-4eb4-b7c5-82a7d27cb2ef None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:02.498178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0235af-1b38-45e9-9756-fb38141f0fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:02.499178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0235af-1b38-45e9-9756-fb38141f0fcf None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.499657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0235af-1b38-45e9-9756-fb38141f0fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.500127 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0235af-1b38-45e9-9756-fb38141f0fcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.511763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0235af-1b38-45e9-9756-fb38141f0fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:02.513543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0235af-1b38-45e9-9756-fb38141f0fcf None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:02.522907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:02.526447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37307df0-a867-4dbb-887a-3d7d4d68dee7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-de0235af-1b38-45e9-9756-fb38141f0fcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:02.531202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:02.538493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:02.539391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:02.539391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:02.539391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:02.540304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 318/639] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:02.541578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0235af-1b38-45e9-9756-fb38141f0fcf None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:02.562987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb5a56-97aa-491e-9bc9-20a69970fa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:02.563619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb5a56-97aa-491e-9bc9-20a69970fa98 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.563900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb5a56-97aa-491e-9bc9-20a69970fa98 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.564175 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb5a56-97aa-491e-9bc9-20a69970fa98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.576951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb5a56-97aa-491e-9bc9-20a69970fa98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:02.578418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb5a56-97aa-491e-9bc9-20a69970fa98 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-fbbb5a56-97aa-491e-9bc9-20a69970fa98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:02.595373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:02.596422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:02.597326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:02.597326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:02.597326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:02.597326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 322/640] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:02.602216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb5a56-97aa-491e-9bc9-20a69970fa98 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:02.659299 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0235af-1b38-45e9-9756-fb38141f0fcf None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:02.692240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:02.697969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e06bac-85b5-4903-96ab-553cb5235a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:02.698575 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06bac-85b5-4903-96ab-553cb5235a65 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.698851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06bac-85b5-4903-96ab-553cb5235a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.699120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06bac-85b5-4903-96ab-553cb5235a65 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.699777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e06bac-85b5-4903-96ab-553cb5235a65 None admin] RBAC: Adding query filter params (name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:02.700076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e06bac-85b5-4903-96ab-553cb5235a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:02.701464 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e06bac-85b5-4903-96ab-553cb5235a65 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:02.710581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb5a56-97aa-491e-9bc9-20a69970fa98 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:02.715226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 319/641] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:58:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:02.716158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e06bac-85b5-4903-96ab-553cb5235a65 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:02.748416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5372f35-ef9f-41af-a662-daab12320857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:02.749049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5372f35-ef9f-41af-a662-daab12320857 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.749325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5372f35-ef9f-41af-a662-daab12320857 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.749632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5372f35-ef9f-41af-a662-daab12320857 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.750268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5372f35-ef9f-41af-a662-daab12320857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:02.750589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5372f35-ef9f-41af-a662-daab12320857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:02.752014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5372f35-ef9f-41af-a662-daab12320857 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:02.765720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 323/642] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:02.766368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5372f35-ef9f-41af-a662-daab12320857 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:02.782757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:02.839360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e06bac-85b5-4903-96ab-553cb5235a65 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:02.875150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5372f35-ef9f-41af-a662-daab12320857 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:02.880968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5bb00-3663-424c-9523-1ef2c2e598d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:02.881806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5bb00-3663-424c-9523-1ef2c2e598d5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.882227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5bb00-3663-424c-9523-1ef2c2e598d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.882641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5bb00-3663-424c-9523-1ef2c2e598d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.883449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5bb00-3663-424c-9523-1ef2c2e598d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:02.884004 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5bb00-3663-424c-9523-1ef2c2e598d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:02.885536 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5bb00-3663-424c-9523-1ef2c2e598d5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:02.898198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 320/643] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:02.899713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5bb00-3663-424c-9523-1ef2c2e598d5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:02.918679 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65765ba-a0f3-433f-b338-47a1ef5c1232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:02.919426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65765ba-a0f3-433f-b338-47a1ef5c1232 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.919708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65765ba-a0f3-433f-b338-47a1ef5c1232 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.919982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65765ba-a0f3-433f-b338-47a1ef5c1232 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.932264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65765ba-a0f3-433f-b338-47a1ef5c1232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-f65765ba-a0f3-433f-b338-47a1ef5c1232 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:02.937213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:02.938192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:02.939074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:02.943669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 324/644] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:58:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:02.944444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65765ba-a0f3-433f-b338-47a1ef5c1232 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:02.966058 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:03.018554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:03.023219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5bb00-3663-424c-9523-1ef2c2e598d5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:03.048515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65765ba-a0f3-433f-b338-47a1ef5c1232 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:03.062853 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34c373-8361-4d3c-b6eb-f6aaeeef547a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:03.063753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34c373-8361-4d3c-b6eb-f6aaeeef547a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.064247 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34c373-8361-4d3c-b6eb-f6aaeeef547a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.064796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34c373-8361-4d3c-b6eb-f6aaeeef547a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.078910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34c373-8361-4d3c-b6eb-f6aaeeef547a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:03.080580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34c373-8361-4d3c-b6eb-f6aaeeef547a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:03.083711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed71c23-2a5e-45cc-be2e-300fff36803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:03.084341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed71c23-2a5e-45cc-be2e-300fff36803e None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.084677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed71c23-2a5e-45cc-be2e-300fff36803e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.084993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed71c23-2a5e-45cc-be2e-300fff36803e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.092693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed71c23-2a5e-45cc-be2e-300fff36803e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:03.094164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed71c23-2a5e-45cc-be2e-300fff36803e None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-8d34c373-8361-4d3c-b6eb-f6aaeeef547a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:03.097396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:03.098602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:03.099848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:03.099848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:03.099848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:03.101252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 321/645] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:03.102270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34c373-8361-4d3c-b6eb-f6aaeeef547a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-6ed71c23-2a5e-45cc-be2e-300fff36803e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:03.105935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:03.107458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 325/646] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:58:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:03.108483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed71c23-2a5e-45cc-be2e-300fff36803e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:03.212623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed71c23-2a5e-45cc-be2e-300fff36803e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:03.225634 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34c373-8361-4d3c-b6eb-f6aaeeef547a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:03.233188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:03.244206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:03.249128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c6c15-f6af-49f5-bd10-a8dd46bcc3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:03.249798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c6c15-f6af-49f5-bd10-a8dd46bcc3eb None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.250018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c6c15-f6af-49f5-bd10-a8dd46bcc3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.250291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c6c15-f6af-49f5-bd10-a8dd46bcc3eb None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.250954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c6c15-f6af-49f5-bd10-a8dd46bcc3eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:03.251252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c6c15-f6af-49f5-bd10-a8dd46bcc3eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:03.252612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c6c15-f6af-49f5-bd10-a8dd46bcc3eb None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:03.266846 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 326/647] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:58:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:03.267031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31091c99-3899-4ca7-b9de-7a0c74d956ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:03.267031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31091c99-3899-4ca7-b9de-7a0c74d956ce None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.267215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c6c15-f6af-49f5-bd10-a8dd46bcc3eb None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:03.267301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31091c99-3899-4ca7-b9de-7a0c74d956ce None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.267301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31091c99-3899-4ca7-b9de-7a0c74d956ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.275031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31091c99-3899-4ca7-b9de-7a0c74d956ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:03.276490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31091c99-3899-4ca7-b9de-7a0c74d956ce None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-31091c99-3899-4ca7-b9de-7a0c74d956ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:03.289957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:03.298922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:03.299802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:03.299802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:03.299802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:03.299802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:03.299802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:03.299802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:03.299802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:03.299802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:03.299802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 322/648] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:03.299802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31091c99-3899-4ca7-b9de-7a0c74d956ce None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:03.382242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c6c15-f6af-49f5-bd10-a8dd46bcc3eb None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:03.411076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31091c99-3899-4ca7-b9de-7a0c74d956ce None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:03.419957 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153a312-e6be-4aa6-a7a5-eecee0efd3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:03.420681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153a312-e6be-4aa6-a7a5-eecee0efd3ea None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.420985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153a312-e6be-4aa6-a7a5-eecee0efd3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.421258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153a312-e6be-4aa6-a7a5-eecee0efd3ea None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.422021 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153a312-e6be-4aa6-a7a5-eecee0efd3ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:03.422322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153a312-e6be-4aa6-a7a5-eecee0efd3ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:03.423680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153a312-e6be-4aa6-a7a5-eecee0efd3ea None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:03.441641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 327/649] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:03.441641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153a312-e6be-4aa6-a7a5-eecee0efd3ea None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:03.452530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d9a40-d9f1-40a1-bb42-ec13f7486826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:03.453192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d9a40-d9f1-40a1-bb42-ec13f7486826 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.453472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d9a40-d9f1-40a1-bb42-ec13f7486826 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.453746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d9a40-d9f1-40a1-bb42-ec13f7486826 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.454564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d9a40-d9f1-40a1-bb42-ec13f7486826 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:03.454821 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d9a40-d9f1-40a1-bb42-ec13f7486826 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:03.456273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d9a40-d9f1-40a1-bb42-ec13f7486826 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:03.476159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 323/650] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Aug 24 09:58:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:03.477158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d9a40-d9f1-40a1-bb42-ec13f7486826 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:03.504637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:03.508147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:03.555869 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153a312-e6be-4aa6-a7a5-eecee0efd3ea None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:03.588240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d9a40-d9f1-40a1-bb42-ec13f7486826 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:03.592728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b28cd-5920-40f0-a99a-cc032b4d9e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:03.593391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b28cd-5920-40f0-a99a-cc032b4d9e5f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.593675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b28cd-5920-40f0-a99a-cc032b4d9e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.593950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b28cd-5920-40f0-a99a-cc032b4d9e5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.606924 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b28cd-5920-40f0-a99a-cc032b4d9e5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:03.608389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b28cd-5920-40f0-a99a-cc032b4d9e5f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-1d8b28cd-5920-40f0-a99a-cc032b4d9e5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:03.623020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:03.624014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:03.625541 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:03.625541 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:03.625541 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:03.625541 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 328/651] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:03.625541 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b28cd-5920-40f0-a99a-cc032b4d9e5f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:03.631413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb35bc7-0387-4cfe-9cc0-645c1680cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:03.632033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb35bc7-0387-4cfe-9cc0-645c1680cf19 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.632316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb35bc7-0387-4cfe-9cc0-645c1680cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.632590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb35bc7-0387-4cfe-9cc0-645c1680cf19 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.633368 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb35bc7-0387-4cfe-9cc0-645c1680cf19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:03.633671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb35bc7-0387-4cfe-9cc0-645c1680cf19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:03.635102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb35bc7-0387-4cfe-9cc0-645c1680cf19 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:03.648547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 324/652] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:03.649527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb35bc7-0387-4cfe-9cc0-645c1680cf19 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:03.738000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:03.739961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b28cd-5920-40f0-a99a-cc032b4d9e5f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:03.747067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:03.754969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb35bc7-0387-4cfe-9cc0-645c1680cf19 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:03.775052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cf453-f144-4952-9c27-2eb97c001260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:03.775682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cf453-f144-4952-9c27-2eb97c001260 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.775953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cf453-f144-4952-9c27-2eb97c001260 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.776218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cf453-f144-4952-9c27-2eb97c001260 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.783928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cf453-f144-4952-9c27-2eb97c001260 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:03.785406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cf453-f144-4952-9c27-2eb97c001260 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:03.791246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9540d31-231a-47a5-affa-c9dbb9ddb39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:03.800158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9540d31-231a-47a5-affa-c9dbb9ddb39f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.800158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9540d31-231a-47a5-affa-c9dbb9ddb39f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.800158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9540d31-231a-47a5-affa-c9dbb9ddb39f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-f81cf453-f144-4952-9c27-2eb97c001260 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:03.800531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:03.801530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 329/653] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:58:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:03.802473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cf453-f144-4952-9c27-2eb97c001260 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:03.868488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9540d31-231a-47a5-affa-c9dbb9ddb39f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:03.870013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9540d31-231a-47a5-affa-c9dbb9ddb39f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:03.910886 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cf453-f144-4952-9c27-2eb97c001260 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:03.944664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c60384-1c44-441a-b5e3-e6fcb3c865cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:03.945258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c60384-1c44-441a-b5e3-e6fcb3c865cd None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.945546 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c60384-1c44-441a-b5e3-e6fcb3c865cd None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.945824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c60384-1c44-441a-b5e3-e6fcb3c865cd None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.946465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c60384-1c44-441a-b5e3-e6fcb3c865cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:03.946767 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c60384-1c44-441a-b5e3-e6fcb3c865cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:03.948128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c60384-1c44-441a-b5e3-e6fcb3c865cd None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:03.955609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:03.961033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 330/654] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:03.961867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c60384-1c44-441a-b5e3-e6fcb3c865cd None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:03.977183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 325/655] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:58:03.977183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f164f6-4373-46b9-b70b-2ddbaa28eca9 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:03.977183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f164f6-4373-46b9-b70b-2ddbaa28eca9 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:03.977183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f164f6-4373-46b9-b70b-2ddbaa28eca9 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:03.977183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 326/656] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:03.977183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f164f6-4373-46b9-b70b-2ddbaa28eca9 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:03.998369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:04.069654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c60384-1c44-441a-b5e3-e6fcb3c865cd None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:04.085160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f164f6-4373-46b9-b70b-2ddbaa28eca9 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:04.104368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0c808-7daf-4e34-8f8a-f9d8513029d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:04.105022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0c808-7daf-4e34-8f8a-f9d8513029d0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:04.105311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0c808-7daf-4e34-8f8a-f9d8513029d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:04.105589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0c808-7daf-4e34-8f8a-f9d8513029d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:04.106316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0c808-7daf-4e34-8f8a-f9d8513029d0 None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=7cd6d2291f934fbdb25c410c2c3abcb6) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:04.106848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0c808-7daf-4e34-8f8a-f9d8513029d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:04.108093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0c808-7daf-4e34-8f8a-f9d8513029d0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:04.113925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 331/657] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:03 2022] GET /identity/v3/role_assignments?user.id=7cd6d2291f934fbdb25c410c2c3abcb6&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:04.114708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0c808-7daf-4e34-8f8a-f9d8513029d0 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:04.128562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68276a3-c6ac-4c1c-bf2b-17418c12c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:04.130052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68276a3-c6ac-4c1c-bf2b-17418c12c2be None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:04.130518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68276a3-c6ac-4c1c-bf2b-17418c12c2be None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:04.131006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68276a3-c6ac-4c1c-bf2b-17418c12c2be None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:04.131781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68276a3-c6ac-4c1c-bf2b-17418c12c2be None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:04.132091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68276a3-c6ac-4c1c-bf2b-17418c12c2be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:04.133778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68276a3-c6ac-4c1c-bf2b-17418c12c2be None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:04.160222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 327/658] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Aug 24 09:58:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:04.160222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68276a3-c6ac-4c1c-bf2b-17418c12c2be None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:04.213758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:04.240404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:04.251203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0c808-7daf-4e34-8f8a-f9d8513029d0 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:04.283560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68276a3-c6ac-4c1c-bf2b-17418c12c2be None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:04.294201 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75604765-2329-4429-9302-8ad2d0c423b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:04.294581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75604765-2329-4429-9302-8ad2d0c423b9 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:04.294750 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75604765-2329-4429-9302-8ad2d0c423b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:04.295288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75604765-2329-4429-9302-8ad2d0c423b9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:04.306944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75604765-2329-4429-9302-8ad2d0c423b9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:04.308821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75604765-2329-4429-9302-8ad2d0c423b9 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-75604765-2329-4429-9302-8ad2d0c423b9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:04.322347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:04.325608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 332/659] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:58:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44647905-90e8-4cbc-86aa-0907cc0a0348 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:04.326783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44647905-90e8-4cbc-86aa-0907cc0a0348 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:04.327664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44647905-90e8-4cbc-86aa-0907cc0a0348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:04.333924 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7752fd0-df56-4709-9299-808e8bc068ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:04.334669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7752fd0-df56-4709-9299-808e8bc068ac None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:04.336171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7752fd0-df56-4709-9299-808e8bc068ac None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:04.336510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7752fd0-df56-4709-9299-808e8bc068ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:04.344582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-44647905-90e8-4cbc-86aa-0907cc0a0348 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:04.346004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-44647905-90e8-4cbc-86aa-0907cc0a0348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:04.351343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7752fd0-df56-4709-9299-808e8bc068ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:04.353161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7752fd0-df56-4709-9299-808e8bc068ac None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-e7752fd0-df56-4709-9299-808e8bc068ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:04.370493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:04.371541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:04.372417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:04.372417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:04.372417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:04.372417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 328/660] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:04.373129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7752fd0-df56-4709-9299-808e8bc068ac None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:04.422618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 333/661] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:58:04.423596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44647905-90e8-4cbc-86aa-0907cc0a0348 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:04.505783 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:04.517034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7752fd0-df56-4709-9299-808e8bc068ac None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:04.521491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:04.541658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44647905-90e8-4cbc-86aa-0907cc0a0348 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:04.563384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4059ec-9f64-4a48-b040-6156774feb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:04.564084 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4059ec-9f64-4a48-b040-6156774feb1c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:04.564911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4059ec-9f64-4a48-b040-6156774feb1c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:04.565206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4059ec-9f64-4a48-b040-6156774feb1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:04.580066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554facb3-7537-4e6e-98c3-0fd28f1c2140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:04.580902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554facb3-7537-4e6e-98c3-0fd28f1c2140 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:04.581261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554facb3-7537-4e6e-98c3-0fd28f1c2140 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:04.581604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554facb3-7537-4e6e-98c3-0fd28f1c2140 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:04.582426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554facb3-7537-4e6e-98c3-0fd28f1c2140 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:04.583049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554facb3-7537-4e6e-98c3-0fd28f1c2140 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:04.584926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554facb3-7537-4e6e-98c3-0fd28f1c2140 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:04.605687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 334/662] 10.222.0.30 () {56 vars in 1139 bytes} [Wed Aug 24 09:58:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:04.606706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554facb3-7537-4e6e-98c3-0fd28f1c2140 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:04.652589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4059ec-9f64-4a48-b040-6156774feb1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:04.654576 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4059ec-9f64-4a48-b040-6156774feb1c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:04.724804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554facb3-7537-4e6e-98c3-0fd28f1c2140 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:04.766810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a03ee-1590-471b-999e-85e2531cab7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:04.767344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a03ee-1590-471b-999e-85e2531cab7f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:04.767627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a03ee-1590-471b-999e-85e2531cab7f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:04.767904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a03ee-1590-471b-999e-85e2531cab7f None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:04.768544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a03ee-1590-471b-999e-85e2531cab7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:04.768847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a03ee-1590-471b-999e-85e2531cab7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:04.771057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a03ee-1590-471b-999e-85e2531cab7f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:04.774246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:04.778060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:04.792650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 329/663] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:58:04.792839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 335/664] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:04.792947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4059ec-9f64-4a48-b040-6156774feb1c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:04.793593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a03ee-1590-471b-999e-85e2531cab7f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:04.890413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a03ee-1590-471b-999e-85e2531cab7f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:04.909711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4059ec-9f64-4a48-b040-6156774feb1c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:04.927327 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ec282-b8ea-45c7-bfc6-c6ba36058485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:04.927952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec282-b8ea-45c7-bfc6-c6ba36058485 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:04.928219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec282-b8ea-45c7-bfc6-c6ba36058485 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:04.928492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec282-b8ea-45c7-bfc6-c6ba36058485 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:04.939516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ec282-b8ea-45c7-bfc6-c6ba36058485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-508ec282-b8ea-45c7-bfc6-c6ba36058485 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:04.945231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:04.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 336/665] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:58:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:04.947028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ec282-b8ea-45c7-bfc6-c6ba36058485 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:04.954975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b33f43-c276-418f-af52-137ee2a32824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:04.955593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b33f43-c276-418f-af52-137ee2a32824 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:04.955874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b33f43-c276-418f-af52-137ee2a32824 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:04.956407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b33f43-c276-418f-af52-137ee2a32824 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.010268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:05.026828 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:05.034143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b33f43-c276-418f-af52-137ee2a32824 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:05.035654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b33f43-c276-418f-af52-137ee2a32824 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:05.055549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ec282-b8ea-45c7-bfc6-c6ba36058485 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:05.093129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3431d6-f23e-4138-9402-e5624d0616ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:05.093765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3431d6-f23e-4138-9402-e5624d0616ae None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.094045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3431d6-f23e-4138-9402-e5624d0616ae None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.094319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3431d6-f23e-4138-9402-e5624d0616ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.105789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3431d6-f23e-4138-9402-e5624d0616ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:05.107240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3431d6-f23e-4138-9402-e5624d0616ae None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-3f3431d6-f23e-4138-9402-e5624d0616ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:05.125232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:05.126213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:05.127108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:05.127108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:05.127108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:05.127108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 337/666] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:05.127108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3431d6-f23e-4138-9402-e5624d0616ae None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:05.159851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 330/667] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:58:05.162366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b33f43-c276-418f-af52-137ee2a32824 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:05.249222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3431d6-f23e-4138-9402-e5624d0616ae None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:05.259089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:05.261672 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:05.266548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b33f43-c276-418f-af52-137ee2a32824 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:05.291255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb60aa-f117-4b1b-ada8-0cc4a83c8199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:05.292170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb60aa-f117-4b1b-ada8-0cc4a83c8199 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.292620 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb60aa-f117-4b1b-ada8-0cc4a83c8199 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.293119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb60aa-f117-4b1b-ada8-0cc4a83c8199 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.301543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb60aa-f117-4b1b-ada8-0cc4a83c8199 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:05.303030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb60aa-f117-4b1b-ada8-0cc4a83c8199 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:05.309476 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62e1df-33f3-4fa1-8635-d6ab4b38656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:05.310103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62e1df-33f3-4fa1-8635-d6ab4b38656c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.310378 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62e1df-33f3-4fa1-8635-d6ab4b38656c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.310648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62e1df-33f3-4fa1-8635-d6ab4b38656c None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.311328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62e1df-33f3-4fa1-8635-d6ab4b38656c None admin] RBAC: Adding query filter params (name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:05.311626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62e1df-33f3-4fa1-8635-d6ab4b38656c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:05.313051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62e1df-33f3-4fa1-8635-d6ab4b38656c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-c8cb60aa-f117-4b1b-ada8-0cc4a83c8199 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:05.315696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:05.317086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:05.318018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:05.318018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:05.318018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:05.318018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:05.318018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:05.318018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:05.318018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:05.318018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:05.318018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 338/668] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:05.318503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb60aa-f117-4b1b-ada8-0cc4a83c8199 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:05.325889 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 331/669] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:58:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:05.326732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62e1df-33f3-4fa1-8635-d6ab4b38656c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:05.427870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb60aa-f117-4b1b-ada8-0cc4a83c8199 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:05.443502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62e1df-33f3-4fa1-8635-d6ab4b38656c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:05.469591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f085b-6321-41ff-9162-c1476c48fefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:05.470203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f085b-6321-41ff-9162-c1476c48fefe None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.470638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f085b-6321-41ff-9162-c1476c48fefe None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.470915 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f085b-6321-41ff-9162-c1476c48fefe None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.471870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f085b-6321-41ff-9162-c1476c48fefe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:05.472429 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f085b-6321-41ff-9162-c1476c48fefe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:05.474520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f085b-6321-41ff-9162-c1476c48fefe None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:05.478620 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b593-6633-4814-be66-5d187f705598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:05.479248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b593-6633-4814-be66-5d187f705598 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.479532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b593-6633-4814-be66-5d187f705598 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.479804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b593-6633-4814-be66-5d187f705598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.492430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 339/670] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:05.492430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f085b-6321-41ff-9162-c1476c48fefe None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:05.492688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236b593-6633-4814-be66-5d187f705598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:05.492688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236b593-6633-4814-be66-5d187f705598 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b236b593-6633-4814-be66-5d187f705598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:05.507674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:05.508660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:05.510271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:05.510271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:05.510271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:05.510271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 332/671] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:05.510271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b593-6633-4814-be66-5d187f705598 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:05.525157 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:05.530304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:05.605200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f085b-6321-41ff-9162-c1476c48fefe None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:05.622223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b593-6633-4814-be66-5d187f705598 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:05.641304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7d8a1-8bc6-457e-8b39-fbe6ee364681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:05.641916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7d8a1-8bc6-457e-8b39-fbe6ee364681 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.642196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7d8a1-8bc6-457e-8b39-fbe6ee364681 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.642470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7d8a1-8bc6-457e-8b39-fbe6ee364681 None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.643163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7d8a1-8bc6-457e-8b39-fbe6ee364681 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:05.643467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7d8a1-8bc6-457e-8b39-fbe6ee364681 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:05.645563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7d8a1-8bc6-457e-8b39-fbe6ee364681 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:05.658060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 340/672] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:05.658916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7d8a1-8bc6-457e-8b39-fbe6ee364681 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:05.665168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69753f1-8dfa-4af9-b867-09de41d928fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:05.665818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69753f1-8dfa-4af9-b867-09de41d928fe None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.666134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69753f1-8dfa-4af9-b867-09de41d928fe None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.666360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69753f1-8dfa-4af9-b867-09de41d928fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.678760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69753f1-8dfa-4af9-b867-09de41d928fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:05.680226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69753f1-8dfa-4af9-b867-09de41d928fe None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-f69753f1-8dfa-4af9-b867-09de41d928fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:05.695526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:05.696554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:05.697495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:05.697495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:05.697495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:05.697495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 333/673] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:05.697495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69753f1-8dfa-4af9-b867-09de41d928fe None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:05.764141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:05.774368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:05.784416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7d8a1-8bc6-457e-8b39-fbe6ee364681 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:05.810773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69753f1-8dfa-4af9-b867-09de41d928fe None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:05.820601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631d5a2-63af-4a69-a436-c8c6b2653947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:05.821288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631d5a2-63af-4a69-a436-c8c6b2653947 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.821553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631d5a2-63af-4a69-a436-c8c6b2653947 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.821829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631d5a2-63af-4a69-a436-c8c6b2653947 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.829047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631d5a2-63af-4a69-a436-c8c6b2653947 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:05.830646 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631d5a2-63af-4a69-a436-c8c6b2653947 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-5631d5a2-63af-4a69-a436-c8c6b2653947 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:05.843398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:05.844379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:05.845436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:05.845436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:05.845436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:05.845436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:05.845436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:05.845436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:05.845436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:05.845436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:05.845436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 341/674] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:05.845436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631d5a2-63af-4a69-a436-c8c6b2653947 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:05.858219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4419a-66c1-4279-ab90-620789f9c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:05.859032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4419a-66c1-4279-ab90-620789f9c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.859446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4419a-66c1-4279-ab90-620789f9c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.859962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4419a-66c1-4279-ab90-620789f9c4d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.861296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4419a-66c1-4279-ab90-620789f9c4d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:05.861296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4419a-66c1-4279-ab90-620789f9c4d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:05.862949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4419a-66c1-4279-ab90-620789f9c4d4 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:05.885351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 334/675] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:05.885351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4419a-66c1-4279-ab90-620789f9c4d4 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:05.975009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631d5a2-63af-4a69-a436-c8c6b2653947 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:05.997710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4419a-66c1-4279-ab90-620789f9c4d4 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:06.014462 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a95a5-073a-4ebc-bc9b-b1b06009b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:06.015088 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a95a5-073a-4ebc-bc9b-b1b06009b6cc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.015349 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a95a5-073a-4ebc-bc9b-b1b06009b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.015620 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a95a5-073a-4ebc-bc9b-b1b06009b6cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.016339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a95a5-073a-4ebc-bc9b-b1b06009b6cc None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=d0b124ca267b4a4391557ad1396524cf) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:06.016697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a95a5-073a-4ebc-bc9b-b1b06009b6cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:06.018151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a95a5-073a-4ebc-bc9b-b1b06009b6cc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:06.021031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:06.026226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 342/676] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:05 2022] GET /identity/v3/role_assignments?user.id=d0b124ca267b4a4391557ad1396524cf&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:06.026845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a95a5-073a-4ebc-bc9b-b1b06009b6cc None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:06.060837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea8c95-de88-4ea7-a2c9-aa6e22a0ac9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:06.061490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:06.061905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea8c95-de88-4ea7-a2c9-aa6e22a0ac9f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.062194 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea8c95-de88-4ea7-a2c9-aa6e22a0ac9f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.062721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea8c95-de88-4ea7-a2c9-aa6e22a0ac9f None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.063880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea8c95-de88-4ea7-a2c9-aa6e22a0ac9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:06.064241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea8c95-de88-4ea7-a2c9-aa6e22a0ac9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:06.066517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea8c95-de88-4ea7-a2c9-aa6e22a0ac9f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:06.084502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 335/677] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:06.085428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea8c95-de88-4ea7-a2c9-aa6e22a0ac9f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:06.175925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a95a5-073a-4ebc-bc9b-b1b06009b6cc None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:06.217076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea8c95-de88-4ea7-a2c9-aa6e22a0ac9f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:06.217285 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79231843-4d6e-4581-b39a-dfc7860fd8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:06.217746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79231843-4d6e-4581-b39a-dfc7860fd8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.218026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79231843-4d6e-4581-b39a-dfc7860fd8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.218299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79231843-4d6e-4581-b39a-dfc7860fd8d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.218984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79231843-4d6e-4581-b39a-dfc7860fd8d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:06.219287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79231843-4d6e-4581-b39a-dfc7860fd8d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:06.220712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79231843-4d6e-4581-b39a-dfc7860fd8d0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:06.236686 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 343/678] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:06.237984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79231843-4d6e-4581-b39a-dfc7860fd8d0 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:06.260921 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adfe5b-4d10-4e37-8bbf-1e415663b1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:06.261559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adfe5b-4d10-4e37-8bbf-1e415663b1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.261839 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adfe5b-4d10-4e37-8bbf-1e415663b1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.262107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adfe5b-4d10-4e37-8bbf-1e415663b1a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.271859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adfe5b-4d10-4e37-8bbf-1e415663b1a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:06.273536 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adfe5b-4d10-4e37-8bbf-1e415663b1a9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-85adfe5b-4d10-4e37-8bbf-1e415663b1a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:06.286838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:06.288277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:06.289333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:06.289333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:06.289333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:06.289333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:06.289333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:06.289333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:06.289333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:06.289333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:06.289979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 336/679] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:06.292022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d02aa-71f9-4e42-b244-33f70ae8c60a None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.292490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d02aa-71f9-4e42-b244-33f70ae8c60a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.296398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d02aa-71f9-4e42-b244-33f70ae8c60a None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.296398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 337/680] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:06.296950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d02aa-71f9-4e42-b244-33f70ae8c60a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:06.336469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:06.349353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:06.364982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79231843-4d6e-4581-b39a-dfc7860fd8d0 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:06.413566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35082cb4-4fed-4c99-91dc-7034dfa3790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:06.414218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35082cb4-4fed-4c99-91dc-7034dfa3790d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.414502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35082cb4-4fed-4c99-91dc-7034dfa3790d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.414781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35082cb4-4fed-4c99-91dc-7034dfa3790d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.415744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d02aa-71f9-4e42-b244-33f70ae8c60a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:06.423109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35082cb4-4fed-4c99-91dc-7034dfa3790d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:06.424557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35082cb4-4fed-4c99-91dc-7034dfa3790d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-35082cb4-4fed-4c99-91dc-7034dfa3790d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:06.438291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:06.439291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:06.440244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:06.440244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:06.440244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:06.440244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:06.440244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:06.440244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:06.440244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:06.440244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:06.440244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:58:06.440244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:06.440244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 344/681] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:06.440975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67da739-6647-45ad-b095-4c6c0c7bd65a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.440975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67da739-6647-45ad-b095-4c6c0c7bd65a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.441688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67da739-6647-45ad-b095-4c6c0c7bd65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.460610 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-d67da739-6647-45ad-b095-4c6c0c7bd65a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:06.461837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d67da739-6647-45ad-b095-4c6c0c7bd65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:06.463079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496bb55-9f6d-44a3-bafb-1f92ccd56f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:06.463698 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496bb55-9f6d-44a3-bafb-1f92ccd56f0d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.463981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496bb55-9f6d-44a3-bafb-1f92ccd56f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.464274 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496bb55-9f6d-44a3-bafb-1f92ccd56f0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.465038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496bb55-9f6d-44a3-bafb-1f92ccd56f0d None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=c71783d1327a47149a74414218dd7219) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:06.465491 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496bb55-9f6d-44a3-bafb-1f92ccd56f0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:06.466972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496bb55-9f6d-44a3-bafb-1f92ccd56f0d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:06.481308 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 338/682] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:06 2022] GET /identity/v3/role_assignments?user.id=c71783d1327a47149a74414218dd7219&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:06.482796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496bb55-9f6d-44a3-bafb-1f92ccd56f0d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:06.557371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 345/683] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:58:06.558829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67da739-6647-45ad-b095-4c6c0c7bd65a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:06.618917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496bb55-9f6d-44a3-bafb-1f92ccd56f0d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:06.627983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:06.657789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:06.658963 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b1baf-b1e3-45b9-998e-4c7a190cf09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:06.659728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b1baf-b1e3-45b9-998e-4c7a190cf09c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.660010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b1baf-b1e3-45b9-998e-4c7a190cf09c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.660340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b1baf-b1e3-45b9-998e-4c7a190cf09c None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.661590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b1baf-b1e3-45b9-998e-4c7a190cf09c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:06.662272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b1baf-b1e3-45b9-998e-4c7a190cf09c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:06.663757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b1baf-b1e3-45b9-998e-4c7a190cf09c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:06.690575 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 339/684] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:06.697921 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b1baf-b1e3-45b9-998e-4c7a190cf09c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:06.715067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67da739-6647-45ad-b095-4c6c0c7bd65a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:06.758024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218beab-6a64-4161-8584-84f6337f30f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:06.758709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218beab-6a64-4161-8584-84f6337f30f8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.758989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218beab-6a64-4161-8584-84f6337f30f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.759264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218beab-6a64-4161-8584-84f6337f30f8 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.760105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218beab-6a64-4161-8584-84f6337f30f8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:06.760413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218beab-6a64-4161-8584-84f6337f30f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:06.762023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218beab-6a64-4161-8584-84f6337f30f8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:06.784228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 346/685] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Aug 24 09:58:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:06.790958 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218beab-6a64-4161-8584-84f6337f30f8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:06.846586 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b1baf-b1e3-45b9-998e-4c7a190cf09c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:06.901016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af879be2-504f-4b18-91ce-2dc7a2874793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:06.901645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af879be2-504f-4b18-91ce-2dc7a2874793 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.901924 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af879be2-504f-4b18-91ce-2dc7a2874793 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.902196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af879be2-504f-4b18-91ce-2dc7a2874793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.902933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af879be2-504f-4b18-91ce-2dc7a2874793 None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=ae9b6525009b41288dd918437acee92a) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:06.903518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af879be2-504f-4b18-91ce-2dc7a2874793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:06.905450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af879be2-504f-4b18-91ce-2dc7a2874793 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:06.914585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 340/686] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:06 2022] GET /identity/v3/role_assignments?user.id=ae9b6525009b41288dd918437acee92a&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:06.915515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af879be2-504f-4b18-91ce-2dc7a2874793 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:06.922283 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218beab-6a64-4161-8584-84f6337f30f8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:06.950022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:06.967131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:06.976106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c8f4f-daab-4de9-a90a-64a8cbc52f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:06.980422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c8f4f-daab-4de9-a90a-64a8cbc52f48 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.980422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c8f4f-daab-4de9-a90a-64a8cbc52f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.980422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c8f4f-daab-4de9-a90a-64a8cbc52f48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:07.106363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c8f4f-daab-4de9-a90a-64a8cbc52f48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:07.107890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c8f4f-daab-4de9-a90a-64a8cbc52f48 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:07.116292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af879be2-504f-4b18-91ce-2dc7a2874793 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:07.164240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d098-1146-4cb4-8f7f-822568ea76f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:07.165266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d098-1146-4cb4-8f7f-822568ea76f5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:07.165756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d098-1146-4cb4-8f7f-822568ea76f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:07.166793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d098-1146-4cb4-8f7f-822568ea76f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:07.185031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10d098-1146-4cb4-8f7f-822568ea76f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-aa10d098-1146-4cb4-8f7f-822568ea76f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:07.190212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:07.191168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:07.192932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 341/687] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:07.198273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d098-1146-4cb4-8f7f-822568ea76f5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:07.229026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 347/688] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:58:07.235275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c8f4f-daab-4de9-a90a-64a8cbc52f48 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:07.266666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:07.321211 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:07.345526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d098-1146-4cb4-8f7f-822568ea76f5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:07.394184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c8f4f-daab-4de9-a90a-64a8cbc52f48 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:07.409321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce922-b72f-4130-ac50-d73fa9b6171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:07.409928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce922-b72f-4130-ac50-d73fa9b6171f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:07.410329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce922-b72f-4130-ac50-d73fa9b6171f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:07.410600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce922-b72f-4130-ac50-d73fa9b6171f None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:07.411324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ce922-b72f-4130-ac50-d73fa9b6171f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:07.411622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ce922-b72f-4130-ac50-d73fa9b6171f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:07.413332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ce922-b72f-4130-ac50-d73fa9b6171f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:07.429418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 342/689] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:58:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:07.434629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce922-b72f-4130-ac50-d73fa9b6171f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:07.461345 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef71510-7dac-496d-8c2b-f49837fdd941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:07.461980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef71510-7dac-496d-8c2b-f49837fdd941 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:07.462325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef71510-7dac-496d-8c2b-f49837fdd941 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:07.462595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef71510-7dac-496d-8c2b-f49837fdd941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:07.485803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef71510-7dac-496d-8c2b-f49837fdd941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:07.487394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef71510-7dac-496d-8c2b-f49837fdd941 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-2ef71510-7dac-496d-8c2b-f49837fdd941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:07.508895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:07.509946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:07.510840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:07.510840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:07.510840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:07.510840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 348/690] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:07.518912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef71510-7dac-496d-8c2b-f49837fdd941 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:07.560776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce922-b72f-4130-ac50-d73fa9b6171f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:07.599268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5fbd30-925f-4cd3-ad47-fa5fadc86127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:07.599268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fbd30-925f-4cd3-ad47-fa5fadc86127 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:07.599268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fbd30-925f-4cd3-ad47-fa5fadc86127 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:07.600188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fbd30-925f-4cd3-ad47-fa5fadc86127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:07.624714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5fbd30-925f-4cd3-ad47-fa5fadc86127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:07.626249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5fbd30-925f-4cd3-ad47-fa5fadc86127 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-6d5fbd30-925f-4cd3-ad47-fa5fadc86127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:07.641594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:07.642637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:07.643565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:07.643565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:07.643565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:07.643565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 343/691] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:07.650231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5fbd30-925f-4cd3-ad47-fa5fadc86127 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:07.663158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:07.677252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:07.737652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef71510-7dac-496d-8c2b-f49837fdd941 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:07.818439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5fbd30-925f-4cd3-ad47-fa5fadc86127 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:07.825656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e61eb-fff5-45b2-8366-74e663e90b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:07.826635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e61eb-fff5-45b2-8366-74e663e90b42 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:07.827125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e61eb-fff5-45b2-8366-74e663e90b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:07.827638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e61eb-fff5-45b2-8366-74e663e90b42 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:07.828500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e61eb-fff5-45b2-8366-74e663e90b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:07.829036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e61eb-fff5-45b2-8366-74e663e90b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:07.830614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e61eb-fff5-45b2-8366-74e663e90b42 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:07.858983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d51e95-5018-4bc5-ae0d-73c2aeedc8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:07.859272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 349/692] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:07.864893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51e95-5018-4bc5-ae0d-73c2aeedc8bc None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:07.864893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51e95-5018-4bc5-ae0d-73c2aeedc8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:07.864893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51e95-5018-4bc5-ae0d-73c2aeedc8bc None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:07.866038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d51e95-5018-4bc5-ae0d-73c2aeedc8bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:07.866425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d51e95-5018-4bc5-ae0d-73c2aeedc8bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:07.867399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e61eb-fff5-45b2-8366-74e663e90b42 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:07.867878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d51e95-5018-4bc5-ae0d-73c2aeedc8bc None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:07.887751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 344/693] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:07.892119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d51e95-5018-4bc5-ae0d-73c2aeedc8bc None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:07.992043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e61eb-fff5-45b2-8366-74e663e90b42 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:08.001571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:08.054384 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fbbb1-3ee7-4b2c-9752-1921beb6ba1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:08.055069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fbbb1-3ee7-4b2c-9752-1921beb6ba1d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.055339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fbbb1-3ee7-4b2c-9752-1921beb6ba1d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.055615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fbbb1-3ee7-4b2c-9752-1921beb6ba1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.060264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d51e95-5018-4bc5-ae0d-73c2aeedc8bc None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:08.060835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:08.070432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fbbb1-3ee7-4b2c-9752-1921beb6ba1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:08.074803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fbbb1-3ee7-4b2c-9752-1921beb6ba1d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-bf7fbbb1-3ee7-4b2c-9752-1921beb6ba1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:08.096234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:08.097289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:08.098615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:08.098615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:08.098615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:08.098615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:08.098615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:08.098615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:08.098615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:08.098615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:08.104324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 350/694] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:08.117955 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fbbb1-3ee7-4b2c-9752-1921beb6ba1d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:08.136941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf73ea-c9d4-47d4-aba4-be434a73b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:08.137640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf73ea-c9d4-47d4-aba4-be434a73b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.138222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf73ea-c9d4-47d4-aba4-be434a73b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.138222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf73ea-c9d4-47d4-aba4-be434a73b5b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.146494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf73ea-c9d4-47d4-aba4-be434a73b5b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:08.147977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf73ea-c9d4-47d4-aba4-be434a73b5b9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-eedf73ea-c9d4-47d4-aba4-be434a73b5b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:08.163108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:08.164180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:08.165198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:08.165198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:08.165198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:08.165198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:08.165198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:08.165198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:08.165198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:08.165198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:08.165198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:08.165198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:08.165198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 345/695] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Aug 24 09:58:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:08.172609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf73ea-c9d4-47d4-aba4-be434a73b5b9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:08.328901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf73ea-c9d4-47d4-aba4-be434a73b5b9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:08.344306 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fbbb1-3ee7-4b2c-9752-1921beb6ba1d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:08.354474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:08.371832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc143c5-8653-4ebb-811a-367317a8f333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:08.372449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc143c5-8653-4ebb-811a-367317a8f333 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.372729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc143c5-8653-4ebb-811a-367317a8f333 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.373034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc143c5-8653-4ebb-811a-367317a8f333 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.374019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc143c5-8653-4ebb-811a-367317a8f333 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:08.374367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc143c5-8653-4ebb-811a-367317a8f333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:08.375813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc143c5-8653-4ebb-811a-367317a8f333 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:08.402273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 346/696] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Aug 24 09:58:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:08.402273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cc2c7-e022-458d-bfc4-b0e91eb27520 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.402273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cc2c7-e022-458d-bfc4-b0e91eb27520 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.402273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cc2c7-e022-458d-bfc4-b0e91eb27520 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.403135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488792db-2555-4cdf-84e3-51f2da764b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:08.404129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488792db-2555-4cdf-84e3-51f2da764b0d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.404656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488792db-2555-4cdf-84e3-51f2da764b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.405163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488792db-2555-4cdf-84e3-51f2da764b0d None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.405413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 347/697] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:08.406393 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4cc2c7-e022-458d-bfc4-b0e91eb27520 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:08.406594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488792db-2555-4cdf-84e3-51f2da764b0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:08.407184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488792db-2555-4cdf-84e3-51f2da764b0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:08.408891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488792db-2555-4cdf-84e3-51f2da764b0d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:08.430904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 351/698] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:08.433055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94394458-8267-483c-849f-21fc721e9b76 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.433560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94394458-8267-483c-849f-21fc721e9b76 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.434062 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94394458-8267-483c-849f-21fc721e9b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.457077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-94394458-8267-483c-849f-21fc721e9b76 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:08.459707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-94394458-8267-483c-849f-21fc721e9b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:08.517453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:08.526136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4cc2c7-e022-458d-bfc4-b0e91eb27520 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:08.538849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 352/699] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:58:08.540013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94394458-8267-483c-849f-21fc721e9b76 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:08.600447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d16d8-8c67-4744-a47c-112cd101d8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:08.601769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d16d8-8c67-4744-a47c-112cd101d8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.602964 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d16d8-8c67-4744-a47c-112cd101d8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.604529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d16d8-8c67-4744-a47c-112cd101d8e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.697591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94394458-8267-483c-849f-21fc721e9b76 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:08.734901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:08.735808 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad0455-a10f-4635-a3c3-877fe5f8fbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:08.736687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad0455-a10f-4635-a3c3-877fe5f8fbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.737199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad0455-a10f-4635-a3c3-877fe5f8fbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.737690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad0455-a10f-4635-a3c3-877fe5f8fbf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.738828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad0455-a10f-4635-a3c3-877fe5f8fbf6 None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=88fe81f7b4c04e9f9a26b430b39ccd36) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:08.739406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad0455-a10f-4635-a3c3-877fe5f8fbf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:08.741200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad0455-a10f-4635-a3c3-877fe5f8fbf6 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:08.748308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 353/700] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:08 2022] GET /identity/v3/role_assignments?user.id=88fe81f7b4c04e9f9a26b430b39ccd36&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:08.773831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d16d8-8c67-4744-a47c-112cd101d8e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:08.775660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d16d8-8c67-4744-a47c-112cd101d8e9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:08.860974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad0455-a10f-4635-a3c3-877fe5f8fbf6 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:08.946969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 348/701] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:58:08.953749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d16d8-8c67-4744-a47c-112cd101d8e9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:08.962573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:09.035801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad0455-a10f-4635-a3c3-877fe5f8fbf6 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:09.069963 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:09.081826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a3ca9-66b2-4dc2-8d48-5f0955634e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:09.083297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a3ca9-66b2-4dc2-8d48-5f0955634e03 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.083717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a3ca9-66b2-4dc2-8d48-5f0955634e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.084045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a3ca9-66b2-4dc2-8d48-5f0955634e03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.093543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d16d8-8c67-4744-a47c-112cd101d8e9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:09.095916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a3ca9-66b2-4dc2-8d48-5f0955634e03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:09.097558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a3ca9-66b2-4dc2-8d48-5f0955634e03 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-2c8a3ca9-66b2-4dc2-8d48-5f0955634e03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:09.116372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:09.117499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:09.118420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:09.118420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:09.118420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:09.118420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 354/702] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:09.123814 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a3ca9-66b2-4dc2-8d48-5f0955634e03 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:09.140064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec9837-b6f0-4ff2-a34e-76ea0a51d9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:09.141004 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec9837-b6f0-4ff2-a34e-76ea0a51d9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.141301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec9837-b6f0-4ff2-a34e-76ea0a51d9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.141575 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec9837-b6f0-4ff2-a34e-76ea0a51d9b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.156917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec9837-b6f0-4ff2-a34e-76ea0a51d9b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:09.158945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec9837-b6f0-4ff2-a34e-76ea0a51d9b6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b4ec9837-b6f0-4ff2-a34e-76ea0a51d9b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:09.176910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:09.179154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:09.181363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:09.181363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:09.181363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:09.181363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 349/703] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:09.181363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d1c34-d998-49f7-8d2a-6e6ca92dd314 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.181363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d1c34-d998-49f7-8d2a-6e6ca92dd314 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.181363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d1c34-d998-49f7-8d2a-6e6ca92dd314 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.182132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 350/704] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:09.188180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a049e-1241-4590-95bf-c6a45a6b8dd0 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.188462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a049e-1241-4590-95bf-c6a45a6b8dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.188735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a049e-1241-4590-95bf-c6a45a6b8dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.214073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-868a049e-1241-4590-95bf-c6a45a6b8dd0 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:09.216673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-868a049e-1241-4590-95bf-c6a45a6b8dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:09.262916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:09.272556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a3ca9-66b2-4dc2-8d48-5f0955634e03 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:09.324150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9c123-a62f-4769-9697-9b32b4e5b0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:09.325269 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9c123-a62f-4769-9697-9b32b4e5b0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.325803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9c123-a62f-4769-9697-9b32b4e5b0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.326298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9c123-a62f-4769-9697-9b32b4e5b0e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.326795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 351/705] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:58:09.327498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9c123-a62f-4769-9697-9b32b4e5b0e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:09.327737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a049e-1241-4590-95bf-c6a45a6b8dd0 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:09.328297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9c123-a62f-4769-9697-9b32b4e5b0e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:09.329973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9c123-a62f-4769-9697-9b32b4e5b0e8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:09.348337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 355/706] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:09.358495 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9c123-a62f-4769-9697-9b32b4e5b0e8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:09.438845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:09.492318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a049e-1241-4590-95bf-c6a45a6b8dd0 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:09.507968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9c123-a62f-4769-9697-9b32b4e5b0e8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:09.545621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1e443-51aa-4c4b-90b5-66e6e0f7b75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:09.546483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1e443-51aa-4c4b-90b5-66e6e0f7b75b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.546837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1e443-51aa-4c4b-90b5-66e6e0f7b75b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.547272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1e443-51aa-4c4b-90b5-66e6e0f7b75b None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.548102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1e443-51aa-4c4b-90b5-66e6e0f7b75b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:09.548477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1e443-51aa-4c4b-90b5-66e6e0f7b75b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:09.550064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1e443-51aa-4c4b-90b5-66e6e0f7b75b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:09.551464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357c7a7-0044-4968-ac92-d76031a181cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:09.552299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357c7a7-0044-4968-ac92-d76031a181cb None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.552792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357c7a7-0044-4968-ac92-d76031a181cb None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.553309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357c7a7-0044-4968-ac92-d76031a181cb None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.554172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357c7a7-0044-4968-ac92-d76031a181cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:09.555823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357c7a7-0044-4968-ac92-d76031a181cb None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:09.563643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 352/707] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:09.564981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eeb03-b8de-4592-8f66-2dbb1de8ba1f None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.565271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eeb03-b8de-4592-8f66-2dbb1de8ba1f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.565545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eeb03-b8de-4592-8f66-2dbb1de8ba1f None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.570308 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 353/708] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:09.572851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188eeb03-b8de-4592-8f66-2dbb1de8ba1f None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:09.683003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188eeb03-b8de-4592-8f66-2dbb1de8ba1f None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:09.713793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:09.713976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 356/709] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:58:09 2022] POST /identity/v3/users => generated 254 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:09.715909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af086a9b-832a-4df3-a4da-d5d201e62136 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.716183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af086a9b-832a-4df3-a4da-d5d201e62136 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.716451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af086a9b-832a-4df3-a4da-d5d201e62136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.719401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a6501-b18a-4feb-b708-b474e40e2818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:09.720042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a6501-b18a-4feb-b708-b474e40e2818 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.720324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a6501-b18a-4feb-b708-b474e40e2818 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.720600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a6501-b18a-4feb-b708-b474e40e2818 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.728820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a6501-b18a-4feb-b708-b474e40e2818 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:09.730375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a6501-b18a-4feb-b708-b474e40e2818 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-6b7a6501-b18a-4feb-b708-b474e40e2818 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:09.745266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:09.752396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:09.753345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:09.753345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:09.753345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:09.753345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:09.753345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:09.753345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:09.753345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:09.753345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:09.753345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 354/710] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:09.753345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a6501-b18a-4feb-b708-b474e40e2818 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:09.764090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-af086a9b-832a-4df3-a4da-d5d201e62136 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:09.764090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-af086a9b-832a-4df3-a4da-d5d201e62136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:09.779890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:09.865666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 357/711] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:58:09.866877 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af086a9b-832a-4df3-a4da-d5d201e62136 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:09.903917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a6501-b18a-4feb-b708-b474e40e2818 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:09.949903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e85feb-27ea-4735-b346-2842c61056c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:09.951153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e85feb-27ea-4735-b346-2842c61056c6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.951728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e85feb-27ea-4735-b346-2842c61056c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.952288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e85feb-27ea-4735-b346-2842c61056c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.953059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e85feb-27ea-4735-b346-2842c61056c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:09.955345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e85feb-27ea-4735-b346-2842c61056c6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-34e85feb-27ea-4735-b346-2842c61056c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:09.969881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:09.971160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:58:09.972227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:09.975939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 355/712] 10.222.0.30 () {56 vars in 1129 bytes} [Wed Aug 24 09:58:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:09.981983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e85feb-27ea-4735-b346-2842c61056c6 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:10.006160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af086a9b-832a-4df3-a4da-d5d201e62136 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:10.046986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:10.050728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda708d7-ea65-4b38-b9ec-8ecdce643fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:10.052037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda708d7-ea65-4b38-b9ec-8ecdce643fbe None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:10.052519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda708d7-ea65-4b38-b9ec-8ecdce643fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:10.052979 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda708d7-ea65-4b38-b9ec-8ecdce643fbe None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:10.053878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda708d7-ea65-4b38-b9ec-8ecdce643fbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:10.055789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda708d7-ea65-4b38-b9ec-8ecdce643fbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:10.057450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda708d7-ea65-4b38-b9ec-8ecdce643fbe None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:10.072159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 358/713] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:10.078600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda708d7-ea65-4b38-b9ec-8ecdce643fbe None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:10.090879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:10.119366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e85feb-27ea-4735-b346-2842c61056c6 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:10.168631 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfdcfe-1c00-4782-9bef-7e43a05800d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:10.170061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfdcfe-1c00-4782-9bef-7e43a05800d0 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:10.170569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfdcfe-1c00-4782-9bef-7e43a05800d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:10.171126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfdcfe-1c00-4782-9bef-7e43a05800d0 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:10.172801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfdcfe-1c00-4782-9bef-7e43a05800d0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:10.173361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfdcfe-1c00-4782-9bef-7e43a05800d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:10.174970 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfdcfe-1c00-4782-9bef-7e43a05800d0 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:10.181010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 356/714] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Aug 24 09:58:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:10.182168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfdcfe-1c00-4782-9bef-7e43a05800d0 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:10.220712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda708d7-ea65-4b38-b9ec-8ecdce643fbe None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:10.260467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34162ce3-80df-4513-99c7-ce6db2d30382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:10.261106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34162ce3-80df-4513-99c7-ce6db2d30382 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:10.261388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34162ce3-80df-4513-99c7-ce6db2d30382 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:10.261660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34162ce3-80df-4513-99c7-ce6db2d30382 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:10.262393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34162ce3-80df-4513-99c7-ce6db2d30382 None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=7cd6d2291f934fbdb25c410c2c3abcb6) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:10.262699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34162ce3-80df-4513-99c7-ce6db2d30382 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:10.264234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34162ce3-80df-4513-99c7-ce6db2d30382 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:10.276890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 359/715] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:10 2022] GET /identity/v3/role_assignments?user.id=7cd6d2291f934fbdb25c410c2c3abcb6&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:10.276890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34162ce3-80df-4513-99c7-ce6db2d30382 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:10.320886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfdcfe-1c00-4782-9bef-7e43a05800d0 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:10.342681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:10.375664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:10.413409 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34162ce3-80df-4513-99c7-ce6db2d30382 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:10.422412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c6aac-d7b2-45a5-bfbe-ba93aee40148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:10.423067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6aac-d7b2-45a5-bfbe-ba93aee40148 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:10.423343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6aac-d7b2-45a5-bfbe-ba93aee40148 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:10.423610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6aac-d7b2-45a5-bfbe-ba93aee40148 None admin] PATH_INFO: `/v3/services/network` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:10.424197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c6aac-d7b2-45a5-bfbe-ba93aee40148 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:10.426158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c6aac-d7b2-45a5-bfbe-ba93aee40148 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-153c6aac-d7b2-45a5-bfbe-ba93aee40148 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:10.446382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:10.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:10.448313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 357/716] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:10 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:10.457360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c6aac-d7b2-45a5-bfbe-ba93aee40148 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:10.460963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07304f0-9f9b-48b0-8bd4-5e0185ade93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:10.461205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07304f0-9f9b-48b0-8bd4-5e0185ade93c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:10.461492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07304f0-9f9b-48b0-8bd4-5e0185ade93c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:10.461807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07304f0-9f9b-48b0-8bd4-5e0185ade93c None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:10.462451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07304f0-9f9b-48b0-8bd4-5e0185ade93c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:10.462986 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07304f0-9f9b-48b0-8bd4-5e0185ade93c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:10.464207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07304f0-9f9b-48b0-8bd4-5e0185ade93c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:10.477220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 360/717] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Aug 24 09:58:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:10.625496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c6aac-d7b2-45a5-bfbe-ba93aee40148 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:10.683205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025878a-e572-48ed-bc8a-1ad2814b9909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:10.683832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025878a-e572-48ed-bc8a-1ad2814b9909 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:10.684113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025878a-e572-48ed-bc8a-1ad2814b9909 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:10.684386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025878a-e572-48ed-bc8a-1ad2814b9909 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:10.685053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025878a-e572-48ed-bc8a-1ad2814b9909 None admin] RBAC: Adding query filter params (name=network) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:10.685359 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025878a-e572-48ed-bc8a-1ad2814b9909 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:10.686739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025878a-e572-48ed-bc8a-1ad2814b9909 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:10.695184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 358/718] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:10.701085 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07304f0-9f9b-48b0-8bd4-5e0185ade93c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:10.845612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07304f0-9f9b-48b0-8bd4-5e0185ade93c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:10.903599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c7b21-9a75-4bdb-87c7-5557d5f58f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:10.904697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c7b21-9a75-4bdb-87c7-5557d5f58f73 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:10.905260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c7b21-9a75-4bdb-87c7-5557d5f58f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:10.905743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c7b21-9a75-4bdb-87c7-5557d5f58f73 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:10.906797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c7b21-9a75-4bdb-87c7-5557d5f58f73 None admin] RBAC: Adding query filter params (type=network) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:10.907305 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c7b21-9a75-4bdb-87c7-5557d5f58f73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:10.908922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c7b21-9a75-4bdb-87c7-5557d5f58f73 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:10.911015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:10.916134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 361/719] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:11.178005 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5427d-ca09-4151-8269-82ef9a1dbea1 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:11.178562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5427d-ca09-4151-8269-82ef9a1dbea1 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:11.178846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5427d-ca09-4151-8269-82ef9a1dbea1 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:11.180750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 359/720] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:11.187636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d7d32-4171-4b37-802f-4c73953a9bf3 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:11.188070 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d7d32-4171-4b37-802f-4c73953a9bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:11.188474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d7d32-4171-4b37-802f-4c73953a9bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:11.212978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-5d3d7d32-4171-4b37-802f-4c73953a9bf3 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:11.214206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5d3d7d32-4171-4b37-802f-4c73953a9bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:11.310764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 362/721] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:58:11.772313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5427d-ca09-4151-8269-82ef9a1dbea1 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:11.828470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:11.893325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5427d-ca09-4151-8269-82ef9a1dbea1 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:11.930353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e3126-8e3b-40be-9155-9aabc3c2ae48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:11.930988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e3126-8e3b-40be-9155-9aabc3c2ae48 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:11.931261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e3126-8e3b-40be-9155-9aabc3c2ae48 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:11.931528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e3126-8e3b-40be-9155-9aabc3c2ae48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:11.945961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e3126-8e3b-40be-9155-9aabc3c2ae48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-5a0e3126-8e3b-40be-9155-9aabc3c2ae48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:11.951066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:11.952029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:11.952932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 360/722] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:11.959691 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d7d32-4171-4b37-802f-4c73953a9bf3 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:12.092141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d7d32-4171-4b37-802f-4c73953a9bf3 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:12.132500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:12.144694 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203f768-765d-4cc2-82c5-48014e253ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:12.148325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203f768-765d-4cc2-82c5-48014e253ddb None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:12.148325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203f768-765d-4cc2-82c5-48014e253ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:12.148325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203f768-765d-4cc2-82c5-48014e253ddb None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:12.148325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203f768-765d-4cc2-82c5-48014e253ddb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:12.148325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203f768-765d-4cc2-82c5-48014e253ddb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:12.148325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203f768-765d-4cc2-82c5-48014e253ddb None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:12.160735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 363/723] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:58:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:12.166768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e3126-8e3b-40be-9155-9aabc3c2ae48 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:12.308045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:12.312224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e3126-8e3b-40be-9155-9aabc3c2ae48 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:12.361785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9255f-9fe0-4f6b-85fd-6f4dc81770a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:12.362427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9255f-9fe0-4f6b-85fd-6f4dc81770a1 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:12.362758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9255f-9fe0-4f6b-85fd-6f4dc81770a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:12.363682 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9255f-9fe0-4f6b-85fd-6f4dc81770a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:12.385531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9255f-9fe0-4f6b-85fd-6f4dc81770a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:12.386997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9255f-9fe0-4f6b-85fd-6f4dc81770a1 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-98f9255f-9fe0-4f6b-85fd-6f4dc81770a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:12.405741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:12.406793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:12.408067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:12.408067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:12.408067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:12.408067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 361/724] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:12.414276 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203f768-765d-4cc2-82c5-48014e253ddb None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:12.545994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203f768-765d-4cc2-82c5-48014e253ddb None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:12.610003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdc082-2577-469a-b2ce-57f3bf2ca45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:12.610832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdc082-2577-469a-b2ce-57f3bf2ca45d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:12.611119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdc082-2577-469a-b2ce-57f3bf2ca45d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:12.611391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdc082-2577-469a-b2ce-57f3bf2ca45d None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:12.612098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdc082-2577-469a-b2ce-57f3bf2ca45d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:12.612343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdc082-2577-469a-b2ce-57f3bf2ca45d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:12.613796 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdc082-2577-469a-b2ce-57f3bf2ca45d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:12.630960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 364/725] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:12.631193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4217d8-9592-4bae-b84f-59484f7c4e28 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:12.631193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4217d8-9592-4bae-b84f-59484f7c4e28 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:12.631193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4217d8-9592-4bae-b84f-59484f7c4e28 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:12.631193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 362/726] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:12.634144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdc082-2577-469a-b2ce-57f3bf2ca45d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:12.637605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826230a0-a44f-40bb-aa8c-c5571f46dde0 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:12.638360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826230a0-a44f-40bb-aa8c-c5571f46dde0 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:12.638650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826230a0-a44f-40bb-aa8c-c5571f46dde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:12.657448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-826230a0-a44f-40bb-aa8c-c5571f46dde0 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:12.658623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-826230a0-a44f-40bb-aa8c-c5571f46dde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:12.694970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:12.753270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 363/727] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:58:12.755271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f8ff0-33f8-4ecd-a3f0-74b6a4035f7d None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:12.755557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f8ff0-33f8-4ecd-a3f0-74b6a4035f7d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:12.755827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f8ff0-33f8-4ecd-a3f0-74b6a4035f7d None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:12.757902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 364/728] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:12.766681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03c219-a442-4c98-aa94-ebb7984c43e6 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:12.767304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03c219-a442-4c98-aa94-ebb7984c43e6 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:12.767728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03c219-a442-4c98-aa94-ebb7984c43e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:12.787948 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3c03c219-a442-4c98-aa94-ebb7984c43e6 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:12.790647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3c03c219-a442-4c98-aa94-ebb7984c43e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:12.812061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdc082-2577-469a-b2ce-57f3bf2ca45d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:12.889447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 365/729] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:58:12.896886 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2ceae-c164-4d91-9c83-2ebafb41b525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:12.896886 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2ceae-c164-4d91-9c83-2ebafb41b525 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:12.896886 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2ceae-c164-4d91-9c83-2ebafb41b525 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:12.896886 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2ceae-c164-4d91-9c83-2ebafb41b525 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:12.907739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2ceae-c164-4d91-9c83-2ebafb41b525 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:12.909387 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2ceae-c164-4d91-9c83-2ebafb41b525 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-8cd2ceae-c164-4d91-9c83-2ebafb41b525 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:12.932816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:12.933972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:12.934923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:12.934923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:12.934923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:12.934923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:12.934923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:12.934923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:12.934923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:12.934923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:12.934923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:58:12.934923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:12.934923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 365/730] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:12.940012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03c219-a442-4c98-aa94-ebb7984c43e6 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:12.959114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:13.078455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03c219-a442-4c98-aa94-ebb7984c43e6 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:13.117283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde2710-f9a9-468a-ba5d-0f2ed4563556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:13.118143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2710-f9a9-468a-ba5d-0f2ed4563556 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.118708 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2710-f9a9-468a-ba5d-0f2ed4563556 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.119139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2710-f9a9-468a-ba5d-0f2ed4563556 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.120064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2710-f9a9-468a-ba5d-0f2ed4563556 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:13.120519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2710-f9a9-468a-ba5d-0f2ed4563556 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:13.122281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2710-f9a9-468a-ba5d-0f2ed4563556 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:13.123058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d89b6-543c-453c-ad9e-001d317c124e None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.123506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d89b6-543c-453c-ad9e-001d317c124e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.123595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d89b6-543c-453c-ad9e-001d317c124e None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.124992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 366/731] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:13.125697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d89b6-543c-453c-ad9e-001d317c124e None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:13.146837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 366/732] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Aug 24 09:58:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:13.148653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953ebc4-6a77-4ea6-81ed-fb790368eb7f None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.149447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953ebc4-6a77-4ea6-81ed-fb790368eb7f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.149894 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953ebc4-6a77-4ea6-81ed-fb790368eb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.173903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-c953ebc4-6a77-4ea6-81ed-fb790368eb7f None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:13.176412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c953ebc4-6a77-4ea6-81ed-fb790368eb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:13.243979 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d89b6-543c-453c-ad9e-001d317c124e None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:13.256220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:13.263174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:13.276258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 367/733] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:58:13.277249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953ebc4-6a77-4ea6-81ed-fb790368eb7f None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:13.283600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51be455-7e74-46e9-baa6-0af455e7aba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:13.284273 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51be455-7e74-46e9-baa6-0af455e7aba3 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.284556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51be455-7e74-46e9-baa6-0af455e7aba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.284992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51be455-7e74-46e9-baa6-0af455e7aba3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.299231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51be455-7e74-46e9-baa6-0af455e7aba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-d51be455-7e74-46e9-baa6-0af455e7aba3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:13.304838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:13.311516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 367/734] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:13.312363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51be455-7e74-46e9-baa6-0af455e7aba3 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:13.429497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51be455-7e74-46e9-baa6-0af455e7aba3 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:13.440598 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953ebc4-6a77-4ea6-81ed-fb790368eb7f None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:13.466606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ef692-469d-4d3c-9c08-8c3c5c783ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:13.467349 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ef692-469d-4d3c-9c08-8c3c5c783ccf None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.467632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ef692-469d-4d3c-9c08-8c3c5c783ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.467915 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ef692-469d-4d3c-9c08-8c3c5c783ccf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.478514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ef692-469d-4d3c-9c08-8c3c5c783ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:13.479394 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf62fe2-19f3-455d-8895-db3f82609b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:13.480244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf62fe2-19f3-455d-8895-db3f82609b33 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.480719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf62fe2-19f3-455d-8895-db3f82609b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.481235 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf62fe2-19f3-455d-8895-db3f82609b33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-e54ef692-469d-4d3c-9c08-8c3c5c783ccf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:13.483772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:13.484693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:13.485568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 368/735] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:58:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:13.486261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ef692-469d-4d3c-9c08-8c3c5c783ccf None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:13.520520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:13.569016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:13.573253 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf62fe2-19f3-455d-8895-db3f82609b33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:13.574758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf62fe2-19f3-455d-8895-db3f82609b33 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:13.593960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ef692-469d-4d3c-9c08-8c3c5c783ccf None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:13.628386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92625405-a302-4455-a179-8063a9bdef5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:13.629568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92625405-a302-4455-a179-8063a9bdef5d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.629926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92625405-a302-4455-a179-8063a9bdef5d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.630255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92625405-a302-4455-a179-8063a9bdef5d None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.630972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92625405-a302-4455-a179-8063a9bdef5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:13.631277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92625405-a302-4455-a179-8063a9bdef5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:13.632675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92625405-a302-4455-a179-8063a9bdef5d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:13.640882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 369/736] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:58:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:13.642096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62848-b5be-45f1-bd81-76e5ace8885c None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.642384 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62848-b5be-45f1-bd81-76e5ace8885c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.642691 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62848-b5be-45f1-bd81-76e5ace8885c None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.643912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 370/737] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:13.645028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a62848-b5be-45f1-bd81-76e5ace8885c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:13.682381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 368/738] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:58:13.683494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf62fe2-19f3-455d-8895-db3f82609b33 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:13.739029 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:13.748366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a62848-b5be-45f1-bd81-76e5ace8885c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:13.780699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:13.783785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdf0df-57c6-4d78-8d67-624dcb54173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:13.784392 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdf0df-57c6-4d78-8d67-624dcb54173a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.784493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf62fe2-19f3-455d-8895-db3f82609b33 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:13.784674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdf0df-57c6-4d78-8d67-624dcb54173a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.785061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdf0df-57c6-4d78-8d67-624dcb54173a None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.785733 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fdf0df-57c6-4d78-8d67-624dcb54173a None admin] RBAC: Adding query filter params (name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:13.786950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fdf0df-57c6-4d78-8d67-624dcb54173a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:13.787684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fdf0df-57c6-4d78-8d67-624dcb54173a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:13.795883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 371/739] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:58:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:13.796710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdf0df-57c6-4d78-8d67-624dcb54173a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:13.819736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501c44b-7099-4efb-a4ae-fbdfaff4e84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:13.820343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501c44b-7099-4efb-a4ae-fbdfaff4e84b None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.820624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501c44b-7099-4efb-a4ae-fbdfaff4e84b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.820985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501c44b-7099-4efb-a4ae-fbdfaff4e84b None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.821639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501c44b-7099-4efb-a4ae-fbdfaff4e84b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:13.823017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501c44b-7099-4efb-a4ae-fbdfaff4e84b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:13.902324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdf0df-57c6-4d78-8d67-624dcb54173a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:13.929769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 369/740] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:58:13 2022] POST /identity/v3/services => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:13.931018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154e3e6-9846-43a4-a743-b599a50b9897 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.931402 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154e3e6-9846-43a4-a743-b599a50b9897 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.931681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154e3e6-9846-43a4-a743-b599a50b9897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.960133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-f154e3e6-9846-43a4-a743-b599a50b9897 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:13.961190 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003aa2-7d23-4c0b-8cce-0cf70e01985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:13.961428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f154e3e6-9846-43a4-a743-b599a50b9897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:13.961849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003aa2-7d23-4c0b-8cce-0cf70e01985a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.962132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003aa2-7d23-4c0b-8cce-0cf70e01985a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.962450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003aa2-7d23-4c0b-8cce-0cf70e01985a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.975570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003aa2-7d23-4c0b-8cce-0cf70e01985a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:13.977090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003aa2-7d23-4c0b-8cce-0cf70e01985a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-ea003aa2-7d23-4c0b-8cce-0cf70e01985a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:13.993696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:13.994763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:13.995658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:13.995658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:13.995658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:13.995658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 372/741] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:13.996412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003aa2-7d23-4c0b-8cce-0cf70e01985a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:13.999262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:14.050850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 370/742] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:58:14.051953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154e3e6-9846-43a4-a743-b599a50b9897 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:14.144341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003aa2-7d23-4c0b-8cce-0cf70e01985a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:14.178633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:14.192749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8f5b0-d415-4682-a2c6-92fd8adb231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:14.193533 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8f5b0-d415-4682-a2c6-92fd8adb231e None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.193741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154e3e6-9846-43a4-a743-b599a50b9897 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:14.193847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8f5b0-d415-4682-a2c6-92fd8adb231e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.194074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8f5b0-d415-4682-a2c6-92fd8adb231e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.206396 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8f5b0-d415-4682-a2c6-92fd8adb231e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:14.208273 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8f5b0-d415-4682-a2c6-92fd8adb231e None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-90f8f5b0-d415-4682-a2c6-92fd8adb231e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:14.223832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:14.224824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:14.225974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:14.225974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:14.225974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:14.227610 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 373/743] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:14.228673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8f5b0-d415-4682-a2c6-92fd8adb231e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:14.238502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d38bf-0019-48fa-920d-7cd17adda4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:14.239151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d38bf-0019-48fa-920d-7cd17adda4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.239428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d38bf-0019-48fa-920d-7cd17adda4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.239752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d38bf-0019-48fa-920d-7cd17adda4e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.253185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d38bf-0019-48fa-920d-7cd17adda4e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:14.255519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d38bf-0019-48fa-920d-7cd17adda4e6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-d67d38bf-0019-48fa-920d-7cd17adda4e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:14.271375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:14.272382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:14.273760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:14.273760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:14.273760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:14.273760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 371/744] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:14.273999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d38bf-0019-48fa-920d-7cd17adda4e6 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:14.309524 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:14.347194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8f5b0-d415-4682-a2c6-92fd8adb231e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:14.387525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d38bf-0019-48fa-920d-7cd17adda4e6 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:14.421551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af82b97-5541-4e5a-b983-24efdc507148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:14.422164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af82b97-5541-4e5a-b983-24efdc507148 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.422446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af82b97-5541-4e5a-b983-24efdc507148 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.422665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c804e-fc8f-4ba2-857a-6ea9758f3d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:14.422802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af82b97-5541-4e5a-b983-24efdc507148 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.423269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c804e-fc8f-4ba2-857a-6ea9758f3d6a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.423369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af82b97-5541-4e5a-b983-24efdc507148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:14.423542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c804e-fc8f-4ba2-857a-6ea9758f3d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.423638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af82b97-5541-4e5a-b983-24efdc507148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:14.423811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c804e-fc8f-4ba2-857a-6ea9758f3d6a None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.424447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c804e-fc8f-4ba2-857a-6ea9758f3d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:14.424742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c804e-fc8f-4ba2-857a-6ea9758f3d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:14.425033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af82b97-5541-4e5a-b983-24efdc507148 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:14.427129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c804e-fc8f-4ba2-857a-6ea9758f3d6a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:14.436288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:14.439548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 374/745] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:14.440208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 372/746] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:14.441185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c804e-fc8f-4ba2-857a-6ea9758f3d6a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:14.445219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af82b97-5541-4e5a-b983-24efdc507148 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:14.556949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af82b97-5541-4e5a-b983-24efdc507148 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:14.564054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c804e-fc8f-4ba2-857a-6ea9758f3d6a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:14.572849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:14.595199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d9986-a2d6-4224-8444-479dded9b0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:14.595814 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9986-a2d6-4224-8444-479dded9b0af None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.596099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9986-a2d6-4224-8444-479dded9b0af None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.596396 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9986-a2d6-4224-8444-479dded9b0af None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.597076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d9986-a2d6-4224-8444-479dded9b0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:14.597523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d9986-a2d6-4224-8444-479dded9b0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:14.599023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d9986-a2d6-4224-8444-479dded9b0af None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:14.602717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01ee8a-6618-4662-9666-43ab25e948d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:14.603388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01ee8a-6618-4662-9666-43ab25e948d1 None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.603664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01ee8a-6618-4662-9666-43ab25e948d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.603931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01ee8a-6618-4662-9666-43ab25e948d1 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.604571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01ee8a-6618-4662-9666-43ab25e948d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:14.605954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01ee8a-6618-4662-9666-43ab25e948d1 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:14.615938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 375/747] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:14.615938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d9986-a2d6-4224-8444-479dded9b0af None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:14.701764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 373/748] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:58:14 2022] POST /identity/v3/services => generated 240 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:14.703047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01ee8a-6618-4662-9666-43ab25e948d1 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:14.725248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d9986-a2d6-4224-8444-479dded9b0af None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:14.774871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36f6f0-4207-427a-9590-bd00851cb438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:14.775579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36f6f0-4207-427a-9590-bd00851cb438 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.775860 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36f6f0-4207-427a-9590-bd00851cb438 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.776253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36f6f0-4207-427a-9590-bd00851cb438 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.782647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:14.785601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36f6f0-4207-427a-9590-bd00851cb438 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:14.788246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36f6f0-4207-427a-9590-bd00851cb438 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-5e36f6f0-4207-427a-9590-bd00851cb438 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:14.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:14.817063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:14.818612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:14.818612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:14.818612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:14.818612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:14.818612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:14.818612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:14.818612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:14.818612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:14.820805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 376/749] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:14.821916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36f6f0-4207-427a-9590-bd00851cb438 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:14.841860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01ee8a-6618-4662-9666-43ab25e948d1 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:14.861632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:14.892817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fcca97-5ee9-41fe-b0a5-f2b69b69d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:14.893418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcca97-5ee9-41fe-b0a5-f2b69b69d0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.893723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcca97-5ee9-41fe-b0a5-f2b69b69d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.894002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcca97-5ee9-41fe-b0a5-f2b69b69d0a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.903196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcca97-5ee9-41fe-b0a5-f2b69b69d0a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:14.904743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcca97-5ee9-41fe-b0a5-f2b69b69d0a9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-14fcca97-5ee9-41fe-b0a5-f2b69b69d0a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:14.918029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:14.919036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 374/750] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Aug 24 09:58:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:14.920056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fcca97-5ee9-41fe-b0a5-f2b69b69d0a9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:14.968298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36f6f0-4207-427a-9590-bd00851cb438 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:15.006899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837990fd-6278-4fe1-9acb-e262732790fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:15.007911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837990fd-6278-4fe1-9acb-e262732790fa None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:15.008379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837990fd-6278-4fe1-9acb-e262732790fa None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:15.008837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837990fd-6278-4fe1-9acb-e262732790fa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:15.019006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837990fd-6278-4fe1-9acb-e262732790fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:15.020894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837990fd-6278-4fe1-9acb-e262732790fa None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-837990fd-6278-4fe1-9acb-e262732790fa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:15.033083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:15.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:15.035121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:15.035121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:15.035121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:15.035121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:15.035121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:15.035121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:15.035121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:15.035121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:15.035121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:15.035121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:15.037896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 377/751] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Aug 24 09:58:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:15.039281 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837990fd-6278-4fe1-9acb-e262732790fa None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:15.062947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fcca97-5ee9-41fe-b0a5-f2b69b69d0a9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:15.091248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:15.106541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb45c0-5e3c-4427-ad9c-fc2bb653060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:15.107482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb45c0-5e3c-4427-ad9c-fc2bb653060b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:15.107947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb45c0-5e3c-4427-ad9c-fc2bb653060b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:15.108466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb45c0-5e3c-4427-ad9c-fc2bb653060b None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:15.109523 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb45c0-5e3c-4427-ad9c-fc2bb653060b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:15.110013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb45c0-5e3c-4427-ad9c-fc2bb653060b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:15.111630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb45c0-5e3c-4427-ad9c-fc2bb653060b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:15.126351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 375/752] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:15.127499 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb45c0-5e3c-4427-ad9c-fc2bb653060b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:15.172199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:15.185835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837990fd-6278-4fe1-9acb-e262732790fa None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:15.237082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0961245-5af6-4dd6-b32b-b43a82302434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:15.237888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0961245-5af6-4dd6-b32b-b43a82302434 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:15.238698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0961245-5af6-4dd6-b32b-b43a82302434 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:15.239071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0961245-5af6-4dd6-b32b-b43a82302434 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:15.240016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0961245-5af6-4dd6-b32b-b43a82302434 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:15.240403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0961245-5af6-4dd6-b32b-b43a82302434 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:15.241916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0961245-5af6-4dd6-b32b-b43a82302434 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:15.244679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb45c0-5e3c-4427-ad9c-fc2bb653060b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:15.263286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 378/753] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Aug 24 09:58:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:15.264292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0961245-5af6-4dd6-b32b-b43a82302434 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:15.290257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4bb4d-6e6a-4d31-b176-29c93082fc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:15.290257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4bb4d-6e6a-4d31-b176-29c93082fc74 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:15.290257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4bb4d-6e6a-4d31-b176-29c93082fc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:15.290257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4bb4d-6e6a-4d31-b176-29c93082fc74 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:15.290748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4bb4d-6e6a-4d31-b176-29c93082fc74 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:15.290748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4bb4d-6e6a-4d31-b176-29c93082fc74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:15.294583 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4bb4d-6e6a-4d31-b176-29c93082fc74 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:15.320129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 376/754] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Aug 24 09:58:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:15.321172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4bb4d-6e6a-4d31-b176-29c93082fc74 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:15.372599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:15.385748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0961245-5af6-4dd6-b32b-b43a82302434 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:15.440761 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4bb4d-6e6a-4d31-b176-29c93082fc74 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:15.500121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac7fb-2f19-4962-bad5-24406aab56d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:15.500931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac7fb-2f19-4962-bad5-24406aab56d2 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:15.501318 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac7fb-2f19-4962-bad5-24406aab56d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:15.501696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac7fb-2f19-4962-bad5-24406aab56d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:15.502631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac7fb-2f19-4962-bad5-24406aab56d2 None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=88fe81f7b4c04e9f9a26b430b39ccd36) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:15.502822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427baba-c22d-4e7c-baec-5844dde406bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:15.503268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac7fb-2f19-4962-bad5-24406aab56d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:15.504020 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427baba-c22d-4e7c-baec-5844dde406bb None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:15.504485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427baba-c22d-4e7c-baec-5844dde406bb None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:15.504969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427baba-c22d-4e7c-baec-5844dde406bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:15.505181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac7fb-2f19-4962-bad5-24406aab56d2 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:15.516977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427baba-c22d-4e7c-baec-5844dde406bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:15.517272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 379/755] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:15 2022] GET /identity/v3/role_assignments?user.id=88fe81f7b4c04e9f9a26b430b39ccd36&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:15.518406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac7fb-2f19-4962-bad5-24406aab56d2 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:15.519500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427baba-c22d-4e7c-baec-5844dde406bb None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-6427baba-c22d-4e7c-baec-5844dde406bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:15.537111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:15.538213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:15.539391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:15.539391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:15.539391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:15.541281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 377/756] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:15.550022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427baba-c22d-4e7c-baec-5844dde406bb None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:15.556707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:15.682268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:15.695973 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427baba-c22d-4e7c-baec-5844dde406bb None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:15.732380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac7fb-2f19-4962-bad5-24406aab56d2 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:15.744271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27017f6f-f451-4fca-9adf-47af16f81281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:15.745184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27017f6f-f451-4fca-9adf-47af16f81281 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:15.745613 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27017f6f-f451-4fca-9adf-47af16f81281 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:15.746035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27017f6f-f451-4fca-9adf-47af16f81281 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:15.746832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27017f6f-f451-4fca-9adf-47af16f81281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:15.747296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27017f6f-f451-4fca-9adf-47af16f81281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:15.748831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27017f6f-f451-4fca-9adf-47af16f81281 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:15.764290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 378/757] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:15.771076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27017f6f-f451-4fca-9adf-47af16f81281 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:15.858247 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cd0fa-02ad-46cc-aad8-2bc868a8c180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:15.859130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd0fa-02ad-46cc-aad8-2bc868a8c180 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:15.859410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd0fa-02ad-46cc-aad8-2bc868a8c180 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:15.859683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd0fa-02ad-46cc-aad8-2bc868a8c180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:15.890202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27017f6f-f451-4fca-9adf-47af16f81281 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:15.947855 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17e3aa-c333-4051-ac07-b6c10014b4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:15.948735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e3aa-c333-4051-ac07-b6c10014b4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:15.949209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e3aa-c333-4051-ac07-b6c10014b4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:15.949629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e3aa-c333-4051-ac07-b6c10014b4f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:15.958886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17e3aa-c333-4051-ac07-b6c10014b4f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:15.965876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17e3aa-c333-4051-ac07-b6c10014b4f0 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-0c17e3aa-c333-4051-ac07-b6c10014b4f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:15.985955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:15.987121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:15.989528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:15.989528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:15.989528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:15.989528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:15.989528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:15.989528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:15.989528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:15.989528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:15.998244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 379/758] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:16.004665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17e3aa-c333-4051-ac07-b6c10014b4f0 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:16.027172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cd0fa-02ad-46cc-aad8-2bc868a8c180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:16.028919 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cd0fa-02ad-46cc-aad8-2bc868a8c180 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:16.031721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:16.051971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:16.131699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17e3aa-c333-4051-ac07-b6c10014b4f0 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:16.174996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 380/759] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:58:16.174996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f34db-8403-4151-ac0e-15fee42bb726 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.174996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f34db-8403-4151-ac0e-15fee42bb726 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.174996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f34db-8403-4151-ac0e-15fee42bb726 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.176980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 381/760] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:16.178041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f34db-8403-4151-ac0e-15fee42bb726 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:16.183681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498383bf-bb4e-47cc-80ec-8e5c6f979c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:16.184588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498383bf-bb4e-47cc-80ec-8e5c6f979c9f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.185137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498383bf-bb4e-47cc-80ec-8e5c6f979c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.185619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498383bf-bb4e-47cc-80ec-8e5c6f979c9f None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.186616 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498383bf-bb4e-47cc-80ec-8e5c6f979c9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:16.187141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498383bf-bb4e-47cc-80ec-8e5c6f979c9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:16.189279 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498383bf-bb4e-47cc-80ec-8e5c6f979c9f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:16.209728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 380/761] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:16.211302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210760-39da-4fdc-b6f9-d1c0324975cf None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.211814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210760-39da-4fdc-b6f9-d1c0324975cf None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.212290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210760-39da-4fdc-b6f9-d1c0324975cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.236093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-f6210760-39da-4fdc-b6f9-d1c0324975cf None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:16.238443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f6210760-39da-4fdc-b6f9-d1c0324975cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:16.309977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f34db-8403-4151-ac0e-15fee42bb726 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:16.317679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:16.325003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 381/762] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:16.326194 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210760-39da-4fdc-b6f9-d1c0324975cf None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:16.329408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:16.374149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc8960-b814-4c97-aef5-fc4bcf0e1a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:16.375173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc8960-b814-4c97-aef5-fc4bcf0e1a22 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.375707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc8960-b814-4c97-aef5-fc4bcf0e1a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.376187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc8960-b814-4c97-aef5-fc4bcf0e1a22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.400525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc8960-b814-4c97-aef5-fc4bcf0e1a22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:16.402308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc8960-b814-4c97-aef5-fc4bcf0e1a22 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-d6fc8960-b814-4c97-aef5-fc4bcf0e1a22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:16.428246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:16.429259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:16.430162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:16.430162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:16.430162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:16.438103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 382/763] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:16.440432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210760-39da-4fdc-b6f9-d1c0324975cf None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:16.446515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc8960-b814-4c97-aef5-fc4bcf0e1a22 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:16.488006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9340663-1c31-40fe-a4a3-a1a4677e7d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:16.488669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9340663-1c31-40fe-a4a3-a1a4677e7d4f None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.488995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9340663-1c31-40fe-a4a3-a1a4677e7d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.489274 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9340663-1c31-40fe-a4a3-a1a4677e7d4f None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/7cd6d2291f934fbdb25c410c2c3abcb6/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.493003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9340663-1c31-40fe-a4a3-a1a4677e7d4f None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=7cd6d2291f934fbdb25c410c2c3abcb6, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:16.494747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9340663-1c31-40fe-a4a3-a1a4677e7d4f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:16.583751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc8960-b814-4c97-aef5-fc4bcf0e1a22 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:16.621799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b146d-7c19-402e-8dfc-e75ed7147c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:16.622876 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b146d-7c19-402e-8dfc-e75ed7147c8d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.638258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b146d-7c19-402e-8dfc-e75ed7147c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.638258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b146d-7c19-402e-8dfc-e75ed7147c8d None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.638258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b146d-7c19-402e-8dfc-e75ed7147c8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:16.638258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b146d-7c19-402e-8dfc-e75ed7147c8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:16.638258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b146d-7c19-402e-8dfc-e75ed7147c8d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:16.638664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 382/764] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:58:16 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/7cd6d2291f934fbdb25c410c2c3abcb6/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:58:16.643016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 383/765] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:16.648824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9340663-1c31-40fe-a4a3-a1a4677e7d4f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:16.750263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b146d-7c19-402e-8dfc-e75ed7147c8d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:16.761245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:16.791358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:16.803089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9340663-1c31-40fe-a4a3-a1a4677e7d4f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:16.846243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3026ca-fdd3-4ed1-a58c-dcca2b60b156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:16.846888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3026ca-fdd3-4ed1-a58c-dcca2b60b156 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.847914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3026ca-fdd3-4ed1-a58c-dcca2b60b156 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.848161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3026ca-fdd3-4ed1-a58c-dcca2b60b156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.855212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3026ca-fdd3-4ed1-a58c-dcca2b60b156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:16.856681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3026ca-fdd3-4ed1-a58c-dcca2b60b156 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-1a3026ca-fdd3-4ed1-a58c-dcca2b60b156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:16.869040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:16.870151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:16.871153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:16.871153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:16.871153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:16.871153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:16.871153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:16.871153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:16.871153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:16.871153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:16.871153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 383/766] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:16.871698 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b74af7-5ec3-46e9-93f8-24d1f6e3f8d4 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.871989 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b74af7-5ec3-46e9-93f8-24d1f6e3f8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.872264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b74af7-5ec3-46e9-93f8-24d1f6e3f8d4 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.878671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 384/767] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:16.879473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b74af7-5ec3-46e9-93f8-24d1f6e3f8d4 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:16.890688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b146d-7c19-402e-8dfc-e75ed7147c8d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:16.929789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8b16b-0cb4-4103-9abb-96766f9cf4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:16.930768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b16b-0cb4-4103-9abb-96766f9cf4af None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.931201 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b16b-0cb4-4103-9abb-96766f9cf4af None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.931617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b16b-0cb4-4103-9abb-96766f9cf4af None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.932338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8b16b-0cb4-4103-9abb-96766f9cf4af None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:16.933883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8b16b-0cb4-4103-9abb-96766f9cf4af None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-21d8b16b-0cb4-4103-9abb-96766f9cf4af None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:16.945690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:16.946932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:58:16.947913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:16.950713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 384/768] 10.222.0.30 () {56 vars in 1129 bytes} [Wed Aug 24 09:58:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:16.956989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff6f54-98df-4ca0-bb2a-23bc3db327cf None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.957465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff6f54-98df-4ca0-bb2a-23bc3db327cf None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.957887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff6f54-98df-4ca0-bb2a-23bc3db327cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.982699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1bff6f54-98df-4ca0-bb2a-23bc3db327cf None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:16.984151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1bff6f54-98df-4ca0-bb2a-23bc3db327cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:16.993349 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b74af7-5ec3-46e9-93f8-24d1f6e3f8d4 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:17.015319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:17.031266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a6d28-1a99-42f2-aac9-9aa2d2690e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:17.031940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a6d28-1a99-42f2-aac9-9aa2d2690e1c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.032222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a6d28-1a99-42f2-aac9-9aa2d2690e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.032524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a6d28-1a99-42f2-aac9-9aa2d2690e1c None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.033717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a6d28-1a99-42f2-aac9-9aa2d2690e1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:17.034029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a6d28-1a99-42f2-aac9-9aa2d2690e1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:17.036096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a6d28-1a99-42f2-aac9-9aa2d2690e1c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:17.053910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 385/769] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:17.053910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a6d28-1a99-42f2-aac9-9aa2d2690e1c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:17.066280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 385/770] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:17.067507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff6f54-98df-4ca0-bb2a-23bc3db327cf None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:17.105809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:17.163565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a6d28-1a99-42f2-aac9-9aa2d2690e1c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:17.198821 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7afb08-93c3-4989-a557-4c511d4ba427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:17.199435 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7afb08-93c3-4989-a557-4c511d4ba427 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.199717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7afb08-93c3-4989-a557-4c511d4ba427 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.199996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7afb08-93c3-4989-a557-4c511d4ba427 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.200627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7afb08-93c3-4989-a557-4c511d4ba427 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:17.201014 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7afb08-93c3-4989-a557-4c511d4ba427 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:17.202706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7afb08-93c3-4989-a557-4c511d4ba427 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:17.209309 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 386/771] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Aug 24 09:58:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:17.214641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7afb08-93c3-4989-a557-4c511d4ba427 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:17.302301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff6f54-98df-4ca0-bb2a-23bc3db327cf None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:17.307770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:17.373538 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6bc3-a5e6-45ec-a709-0044128fe722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:17.374523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6bc3-a5e6-45ec-a709-0044128fe722 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.374818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6bc3-a5e6-45ec-a709-0044128fe722 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.375098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6bc3-a5e6-45ec-a709-0044128fe722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.376230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e6bc3-a5e6-45ec-a709-0044128fe722 None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=8b7227998e7846c7a9bc14b12b637ee0) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:17.376552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e6bc3-a5e6-45ec-a709-0044128fe722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:17.379010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e6bc3-a5e6-45ec-a709-0044128fe722 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:17.386659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 386/772] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:17 2022] GET /identity/v3/role_assignments?user.id=8b7227998e7846c7a9bc14b12b637ee0&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:17.387808 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6bc3-a5e6-45ec-a709-0044128fe722 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:17.397129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7afb08-93c3-4989-a557-4c511d4ba427 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:17.485700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e8403-8cb5-4a93-b4cd-c3bcc17c0e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:17.486742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e8403-8cb5-4a93-b4cd-c3bcc17c0e2e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.487238 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e8403-8cb5-4a93-b4cd-c3bcc17c0e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.487751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e8403-8cb5-4a93-b4cd-c3bcc17c0e2e None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.489527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e8403-8cb5-4a93-b4cd-c3bcc17c0e2e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:17.490066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e8403-8cb5-4a93-b4cd-c3bcc17c0e2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:17.491720 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e8403-8cb5-4a93-b4cd-c3bcc17c0e2e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:17.508509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 387/773] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Aug 24 09:58:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:17.514835 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e8403-8cb5-4a93-b4cd-c3bcc17c0e2e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:17.533269 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:17.539577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6bc3-a5e6-45ec-a709-0044128fe722 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:17.583406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5ccc8-88ba-4184-9539-1a0f0a857fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:17.583933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5ccc8-88ba-4184-9539-1a0f0a857fbc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.584216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5ccc8-88ba-4184-9539-1a0f0a857fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.584489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5ccc8-88ba-4184-9539-1a0f0a857fbc None admin] PATH_INFO: `/v3/services/network` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.585099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5ccc8-88ba-4184-9539-1a0f0a857fbc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:17.586663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5ccc8-88ba-4184-9539-1a0f0a857fbc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-31c5ccc8-88ba-4184-9539-1a0f0a857fbc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:17.597732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:17.599052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:58:17.600469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:17.601449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 387/774] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:17 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:17.606803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5ccc8-88ba-4184-9539-1a0f0a857fbc None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:17.655075 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e8403-8cb5-4a93-b4cd-c3bcc17c0e2e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:17.667115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:17.704806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05466664-4afe-4221-baa5-e5778b46565d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:17.705968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05466664-4afe-4221-baa5-e5778b46565d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.706599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05466664-4afe-4221-baa5-e5778b46565d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.707792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05466664-4afe-4221-baa5-e5778b46565d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.708745 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05466664-4afe-4221-baa5-e5778b46565d None admin] RBAC: Adding query filter params (interface=public, service_id=35dc08790c46413f85ea6437377f27d9, region_id=RegionOne) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:17.709375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05466664-4afe-4221-baa5-e5778b46565d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:17.711172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05466664-4afe-4221-baa5-e5778b46565d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:17.720778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 388/775] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Aug 24 09:58:17 2022] GET /identity/v3/endpoints?service_id=35dc08790c46413f85ea6437377f27d9&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:17.725367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5ccc8-88ba-4184-9539-1a0f0a857fbc None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:17.726695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05466664-4afe-4221-baa5-e5778b46565d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:17.763811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577fda7-4d1e-4e47-84bd-89b9450e1608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:17.764536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577fda7-4d1e-4e47-84bd-89b9450e1608 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.764751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577fda7-4d1e-4e47-84bd-89b9450e1608 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.765367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577fda7-4d1e-4e47-84bd-89b9450e1608 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.765900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577fda7-4d1e-4e47-84bd-89b9450e1608 None admin] RBAC: Adding query filter params (name=network) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:17.766198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577fda7-4d1e-4e47-84bd-89b9450e1608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:17.770973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577fda7-4d1e-4e47-84bd-89b9450e1608 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:17.780200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 388/776] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:17.786107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577fda7-4d1e-4e47-84bd-89b9450e1608 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:17.809854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:17.903776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05466664-4afe-4221-baa5-e5778b46565d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:17.906380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577fda7-4d1e-4e47-84bd-89b9450e1608 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:17.942955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019d11e-8761-40f8-82c6-c26bc8f9a957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:17.943599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019d11e-8761-40f8-82c6-c26bc8f9a957 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.943880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019d11e-8761-40f8-82c6-c26bc8f9a957 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.944154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019d11e-8761-40f8-82c6-c26bc8f9a957 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.944760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019d11e-8761-40f8-82c6-c26bc8f9a957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:17.945994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acffc8b-2e24-4fdc-b1a0-3d9d7210a796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:17.946169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019d11e-8761-40f8-82c6-c26bc8f9a957 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:17.946745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acffc8b-2e24-4fdc-b1a0-3d9d7210a796 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.947178 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acffc8b-2e24-4fdc-b1a0-3d9d7210a796 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.947521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acffc8b-2e24-4fdc-b1a0-3d9d7210a796 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.948304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acffc8b-2e24-4fdc-b1a0-3d9d7210a796 None admin] RBAC: Adding query filter params (type=network) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:17.948712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acffc8b-2e24-4fdc-b1a0-3d9d7210a796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:17.950160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acffc8b-2e24-4fdc-b1a0-3d9d7210a796 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:17.955684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 389/777] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:17.958317 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c1fa7-2d0a-4ff4-9855-80b31018df49 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.958707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c1fa7-2d0a-4ff4-9855-80b31018df49 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.959103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c1fa7-2d0a-4ff4-9855-80b31018df49 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.960658 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 389/778] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Aug 24 09:58:17 2022] GET /identity/v3/services => generated 772 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:17.960907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 390/779] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:17.961659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019d11e-8761-40f8-82c6-c26bc8f9a957 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:17.971922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f2f8f-1158-4a1b-b8b7-1bf5cb44467f None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.973215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f2f8f-1158-4a1b-b8b7-1bf5cb44467f None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.973671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f2f8f-1158-4a1b-b8b7-1bf5cb44467f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.997819 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1f7f2f8f-1158-4a1b-b8b7-1bf5cb44467f None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:17.999559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1f7f2f8f-1158-4a1b-b8b7-1bf5cb44467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:18.014422 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:18.077651 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 391/780] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:18.140522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019d11e-8761-40f8-82c6-c26bc8f9a957 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:18.193985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e5c41-4ef3-4d0e-ad63-891ed14bf664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:18.195243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e5c41-4ef3-4d0e-ad63-891ed14bf664 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:18.195838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e5c41-4ef3-4d0e-ad63-891ed14bf664 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:18.196439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e5c41-4ef3-4d0e-ad63-891ed14bf664 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:18.197731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e5c41-4ef3-4d0e-ad63-891ed14bf664 None admin] RBAC: Adding query filter params (interface=public, service_id=75c91dc8b655484ea9bfa8f7cf5e0b50, region_id=RegionOne) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:18.198498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e5c41-4ef3-4d0e-ad63-891ed14bf664 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:18.201317 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e5c41-4ef3-4d0e-ad63-891ed14bf664 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:18.211124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 390/781] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Aug 24 09:58:17 2022] GET /identity/v3/endpoints?service_id=75c91dc8b655484ea9bfa8f7cf5e0b50&interface=public®ion_id=RegionOne => generated 194 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:18.217006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f2f8f-1158-4a1b-b8b7-1bf5cb44467f None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:18.221323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:18.352957 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f2f8f-1158-4a1b-b8b7-1bf5cb44467f None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:18.402344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb2f78-7564-43da-b376-e9eeb530ef6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:18.402970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb2f78-7564-43da-b376-e9eeb530ef6f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:18.403245 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb2f78-7564-43da-b376-e9eeb530ef6f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:18.403515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb2f78-7564-43da-b376-e9eeb530ef6f None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:18.404122 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb2f78-7564-43da-b376-e9eeb530ef6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:18.405523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb2f78-7564-43da-b376-e9eeb530ef6f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:18.412530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 392/782] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Aug 24 09:58:18 2022] GET /identity/v3/services => generated 772 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:18.503371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e5c41-4ef3-4d0e-ad63-891ed14bf664 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:18.621153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:18.624143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e5c41-4ef3-4d0e-ad63-891ed14bf664 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:18.661746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fecf68-6e1d-4cfc-9db3-539659d573ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:18.662607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fecf68-6e1d-4cfc-9db3-539659d573ee None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:18.663241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fecf68-6e1d-4cfc-9db3-539659d573ee None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:18.663515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fecf68-6e1d-4cfc-9db3-539659d573ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:18.681015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fecf68-6e1d-4cfc-9db3-539659d573ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-38fecf68-6e1d-4cfc-9db3-539659d573ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:18.686590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:18.687547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:18.688545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 391/783] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:18.694867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb2f78-7564-43da-b376-e9eeb530ef6f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:18.791114 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:18.842320 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb2f78-7564-43da-b376-e9eeb530ef6f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:18.898063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bf4bb-98b9-4da3-b3e4-767c33a57b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:18.898746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bf4bb-98b9-4da3-b3e4-767c33a57b6f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:18.899034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bf4bb-98b9-4da3-b3e4-767c33a57b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:18.899359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bf4bb-98b9-4da3-b3e4-767c33a57b6f None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:18.900041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bf4bb-98b9-4da3-b3e4-767c33a57b6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:18.900346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bf4bb-98b9-4da3-b3e4-767c33a57b6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:18.901758 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bf4bb-98b9-4da3-b3e4-767c33a57b6f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:18.917140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 393/784] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:58:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:18.922968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fecf68-6e1d-4cfc-9db3-539659d573ee None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:18.994020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60664663-e531-4877-9425-967e158fc786 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:18.994671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60664663-e531-4877-9425-967e158fc786 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:18.995151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60664663-e531-4877-9425-967e158fc786 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:18.998157 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 394/785] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:58:19.003650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc49821-aac4-4ca6-a394-082ce75147ad None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:19.006402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc49821-aac4-4ca6-a394-082ce75147ad None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:19.006402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc49821-aac4-4ca6-a394-082ce75147ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:19.037739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6cc49821-aac4-4ca6-a394-082ce75147ad None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:19.038959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6cc49821-aac4-4ca6-a394-082ce75147ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:19.122811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 395/786] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:19.130635 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fecf68-6e1d-4cfc-9db3-539659d573ee None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:19.178665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d08b1-3325-4860-a1e1-55a37a2d4b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:19.179713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d08b1-3325-4860-a1e1-55a37a2d4b84 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:19.180191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d08b1-3325-4860-a1e1-55a37a2d4b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:19.180679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d08b1-3325-4860-a1e1-55a37a2d4b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:19.195259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d08b1-3325-4860-a1e1-55a37a2d4b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:19.196739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d08b1-3325-4860-a1e1-55a37a2d4b84 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-3f3d08b1-3325-4860-a1e1-55a37a2d4b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:19.222939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:19.223967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:19.225418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:19.225418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:19.225418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:19.225418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 392/787] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:19.230552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc49821-aac4-4ca6-a394-082ce75147ad None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:19.330417 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:19.376813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc49821-aac4-4ca6-a394-082ce75147ad None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:19.418191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8357cb-a011-4be2-a8ee-3fdce1447d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:19.419184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8357cb-a011-4be2-a8ee-3fdce1447d96 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:19.419683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8357cb-a011-4be2-a8ee-3fdce1447d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:19.420153 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8357cb-a011-4be2-a8ee-3fdce1447d96 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:19.421046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8357cb-a011-4be2-a8ee-3fdce1447d96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:19.421564 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8357cb-a011-4be2-a8ee-3fdce1447d96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:19.423133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8357cb-a011-4be2-a8ee-3fdce1447d96 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:19.436139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 396/788] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:19.441976 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d08b1-3325-4860-a1e1-55a37a2d4b84 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:19.471639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:19.535461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8357cb-a011-4be2-a8ee-3fdce1447d96 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:19.651665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8357cb-a011-4be2-a8ee-3fdce1447d96 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:19.655241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d08b1-3325-4860-a1e1-55a37a2d4b84 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:19.675614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:19.692623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6b8d9-5a13-4e89-a08e-1357bad756bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:19.693783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6b8d9-5a13-4e89-a08e-1357bad756bf None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:19.694265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6b8d9-5a13-4e89-a08e-1357bad756bf None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:19.694895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6b8d9-5a13-4e89-a08e-1357bad756bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:19.708330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6b8d9-5a13-4e89-a08e-1357bad756bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-46d6b8d9-5a13-4e89-a08e-1357bad756bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:19.713478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:19.714404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:19.715248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:19.718042 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 397/789] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:19.725052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6b8d9-5a13-4e89-a08e-1357bad756bf None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:19.744329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ad641-386b-4786-a1cb-58b0e3c54fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:19.745689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ad641-386b-4786-a1cb-58b0e3c54fbd None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:19.746164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ad641-386b-4786-a1cb-58b0e3c54fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:19.746629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ad641-386b-4786-a1cb-58b0e3c54fbd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:19.762690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ad641-386b-4786-a1cb-58b0e3c54fbd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:19.764204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ad641-386b-4786-a1cb-58b0e3c54fbd None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-239ad641-386b-4786-a1cb-58b0e3c54fbd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:19.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:19.780449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 393/790] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe32c7d-1553-4163-bb01-7a6b6d5cdc39 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:19.782252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe32c7d-1553-4163-bb01-7a6b6d5cdc39 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:19.783486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe32c7d-1553-4163-bb01-7a6b6d5cdc39 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:19.784960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 394/791] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:19.791171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe32c7d-1553-4163-bb01-7a6b6d5cdc39 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:19.875637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:19.904809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6b8d9-5a13-4e89-a08e-1357bad756bf None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:19.908785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe32c7d-1553-4163-bb01-7a6b6d5cdc39 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:19.953363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a302803-d4e2-4fc4-825e-8988bf52510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:19.953982 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a302803-d4e2-4fc4-825e-8988bf52510f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:19.954202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43055f58-3b97-4fc5-8654-ec77559719b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:19.954441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a302803-d4e2-4fc4-825e-8988bf52510f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:19.954868 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a302803-d4e2-4fc4-825e-8988bf52510f None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:19.955240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43055f58-3b97-4fc5-8654-ec77559719b8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:19.955717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43055f58-3b97-4fc5-8654-ec77559719b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:19.955840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a302803-d4e2-4fc4-825e-8988bf52510f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:19.956083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a302803-d4e2-4fc4-825e-8988bf52510f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:19.956684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43055f58-3b97-4fc5-8654-ec77559719b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:19.957596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43055f58-3b97-4fc5-8654-ec77559719b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:19.957759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a302803-d4e2-4fc4-825e-8988bf52510f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:19.958200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43055f58-3b97-4fc5-8654-ec77559719b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:19.959885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43055f58-3b97-4fc5-8654-ec77559719b8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:19.971723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 398/792] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:58:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:19.974166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a79354-9820-4038-a741-f10a705aa6ce None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:19.974667 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a79354-9820-4038-a741-f10a705aa6ce None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:19.975138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a79354-9820-4038-a741-f10a705aa6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:19.982504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 395/793] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Aug 24 09:58:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:19.983718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a302803-d4e2-4fc4-825e-8988bf52510f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:19.999658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-f4a79354-9820-4038-a741-f10a705aa6ce None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:20.001665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f4a79354-9820-4038-a741-f10a705aa6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:20.053083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:20.089375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 399/794] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:20.090586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a79354-9820-4038-a741-f10a705aa6ce None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:20.106131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a302803-d4e2-4fc4-825e-8988bf52510f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:20.137857 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:20.144955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9b64c-be22-43ca-a117-e8a234fd8415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:20.145689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b64c-be22-43ca-a117-e8a234fd8415 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.146108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b64c-be22-43ca-a117-e8a234fd8415 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.146453 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b64c-be22-43ca-a117-e8a234fd8415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.169129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9b64c-be22-43ca-a117-e8a234fd8415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:20.170972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9b64c-be22-43ca-a117-e8a234fd8415 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-36c9b64c-be22-43ca-a117-e8a234fd8415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:20.196901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:20.197896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:20.198786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:20.198786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:20.198786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:20.198786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 396/795] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:20.200660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9b64c-be22-43ca-a117-e8a234fd8415 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:20.300251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a79354-9820-4038-a741-f10a705aa6ce None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:20.324677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9b64c-be22-43ca-a117-e8a234fd8415 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:20.363978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e90e14-5b42-401a-8c24-41194c41b21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:20.364760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e90e14-5b42-401a-8c24-41194c41b21e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.365150 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e90e14-5b42-401a-8c24-41194c41b21e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.365548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e90e14-5b42-401a-8c24-41194c41b21e None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.366309 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e90e14-5b42-401a-8c24-41194c41b21e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:20.366768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e90e14-5b42-401a-8c24-41194c41b21e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:20.368327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e90e14-5b42-401a-8c24-41194c41b21e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:20.370251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaf294-6d1a-445c-96f6-c7a3bc7de072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:20.370997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaf294-6d1a-445c-96f6-c7a3bc7de072 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.371282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaf294-6d1a-445c-96f6-c7a3bc7de072 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.371556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaf294-6d1a-445c-96f6-c7a3bc7de072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.396081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eaf294-6d1a-445c-96f6-c7a3bc7de072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:20.397672 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eaf294-6d1a-445c-96f6-c7a3bc7de072 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-10eaf294-6d1a-445c-96f6-c7a3bc7de072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:20.429193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:20.430182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:20.431090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:20.431090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:20.431090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:20.431090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 400/796] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:20.437668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaf294-6d1a-445c-96f6-c7a3bc7de072 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:20.489383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:20.505388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 397/797] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:20.506268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e90e14-5b42-401a-8c24-41194c41b21e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:20.560443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:20.578105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaf294-6d1a-445c-96f6-c7a3bc7de072 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:20.615768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3118f-f1cb-49b7-9969-c2448d3354a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:20.616771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3118f-f1cb-49b7-9969-c2448d3354a7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.617102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3118f-f1cb-49b7-9969-c2448d3354a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.617383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3118f-f1cb-49b7-9969-c2448d3354a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.618312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3118f-f1cb-49b7-9969-c2448d3354a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:20.618655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3118f-f1cb-49b7-9969-c2448d3354a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:20.620204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3118f-f1cb-49b7-9969-c2448d3354a7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:20.641625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 401/798] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:20.641625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3118f-f1cb-49b7-9969-c2448d3354a7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:20.645748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e90e14-5b42-401a-8c24-41194c41b21e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:20.689786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b97c8-a021-4c15-83a9-7f4e7bc7cab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:20.691210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b97c8-a021-4c15-83a9-7f4e7bc7cab3 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.691630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b97c8-a021-4c15-83a9-7f4e7bc7cab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.692483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b97c8-a021-4c15-83a9-7f4e7bc7cab3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.706058 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b97c8-a021-4c15-83a9-7f4e7bc7cab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-968b97c8-a021-4c15-83a9-7f4e7bc7cab3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:20.712284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:20.713305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:20.714185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:20.715729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 398/799] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:58:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:20.716976 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b97c8-a021-4c15-83a9-7f4e7bc7cab3 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:20.756081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:20.776714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3118f-f1cb-49b7-9969-c2448d3354a7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:20.814303 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e85d0-6f65-41c6-b5ed-3190e657b010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:20.814931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e85d0-6f65-41c6-b5ed-3190e657b010 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.815255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e85d0-6f65-41c6-b5ed-3190e657b010 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.815533 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e85d0-6f65-41c6-b5ed-3190e657b010 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.823351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e85d0-6f65-41c6-b5ed-3190e657b010 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:20.824824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e85d0-6f65-41c6-b5ed-3190e657b010 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-2b9e85d0-6f65-41c6-b5ed-3190e657b010 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:20.836627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:20.837617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:20.838565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:20.838565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:20.838565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:20.838565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:20.838565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:20.838565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:20.838565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:20.838565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:20.838565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:20.838565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:20.840997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 402/800] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Aug 24 09:58:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:20.841997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e85d0-6f65-41c6-b5ed-3190e657b010 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:20.853707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b97c8-a021-4c15-83a9-7f4e7bc7cab3 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:20.855298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:20.889723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312e053-0125-4024-b521-b71e3a20aacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:20.890352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e053-0125-4024-b521-b71e3a20aacc None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.890632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e053-0125-4024-b521-b71e3a20aacc None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.890911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e053-0125-4024-b521-b71e3a20aacc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.897599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312e053-0125-4024-b521-b71e3a20aacc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:20.899061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312e053-0125-4024-b521-b71e3a20aacc None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-9312e053-0125-4024-b521-b71e3a20aacc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:20.910007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:20.910911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:20.912071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:20.912071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:20.912071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:20.912071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:20.912071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:20.912071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:20.912071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:20.912071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:20.916277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 399/801] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:20.917506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca4d09-3282-475d-ba61-8efffcab9b6c None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.917795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca4d09-3282-475d-ba61-8efffcab9b6c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.918070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca4d09-3282-475d-ba61-8efffcab9b6c None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.919341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 400/802] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:20.920429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef262e3d-bd70-40a1-b1fa-867106ddc753 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.920779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef262e3d-bd70-40a1-b1fa-867106ddc753 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.921078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef262e3d-bd70-40a1-b1fa-867106ddc753 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.925498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 401/803] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:20.925498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef262e3d-bd70-40a1-b1fa-867106ddc753 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:20.951183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e85d0-6f65-41c6-b5ed-3190e657b010 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:20.985224 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3816fe-03fe-47ef-b88c-d232c0702490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:20.985826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3816fe-03fe-47ef-b88c-d232c0702490 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.986107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3816fe-03fe-47ef-b88c-d232c0702490 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.986381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3816fe-03fe-47ef-b88c-d232c0702490 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.987030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3816fe-03fe-47ef-b88c-d232c0702490 None admin] RBAC: Adding query filter params (name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:20.987330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3816fe-03fe-47ef-b88c-d232c0702490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:20.989680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3816fe-03fe-47ef-b88c-d232c0702490 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:21.002928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 403/804] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:58:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:21.002928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3816fe-03fe-47ef-b88c-d232c0702490 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:21.013725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:21.035638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef262e3d-bd70-40a1-b1fa-867106ddc753 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:21.071802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c41d88-c0dc-42c0-bb66-6f1a6e8d422d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:21.072406 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c41d88-c0dc-42c0-bb66-6f1a6e8d422d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:21.072687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c41d88-c0dc-42c0-bb66-6f1a6e8d422d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:21.072990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c41d88-c0dc-42c0-bb66-6f1a6e8d422d None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:21.073754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c41d88-c0dc-42c0-bb66-6f1a6e8d422d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:21.074428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c41d88-c0dc-42c0-bb66-6f1a6e8d422d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:21.075871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c41d88-c0dc-42c0-bb66-6f1a6e8d422d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:21.093703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 402/805] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Aug 24 09:58:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:21.094998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad06871-08fd-4804-9f3f-8b74949d3222 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:21.095293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad06871-08fd-4804-9f3f-8b74949d3222 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:21.095579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad06871-08fd-4804-9f3f-8b74949d3222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:21.110295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3816fe-03fe-47ef-b88c-d232c0702490 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:21.113928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-cad06871-08fd-4804-9f3f-8b74949d3222 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:21.115149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cad06871-08fd-4804-9f3f-8b74949d3222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:21.129846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:21.144753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a87469-bb41-473b-94bb-bf7e1199b0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:21.145402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a87469-bb41-473b-94bb-bf7e1199b0de None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:21.179673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a87469-bb41-473b-94bb-bf7e1199b0de None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:21.179673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a87469-bb41-473b-94bb-bf7e1199b0de None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:21.179673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a87469-bb41-473b-94bb-bf7e1199b0de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:21.179673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a87469-bb41-473b-94bb-bf7e1199b0de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:21.179673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a87469-bb41-473b-94bb-bf7e1199b0de None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:21.179673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 404/806] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:21.180141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d71e6f-b66b-442e-9d00-ca272f7bac76 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:21.180141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d71e6f-b66b-442e-9d00-ca272f7bac76 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:21.180141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d71e6f-b66b-442e-9d00-ca272f7bac76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:21.181506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-54d71e6f-b66b-442e-9d00-ca272f7bac76 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:21.182749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-54d71e6f-b66b-442e-9d00-ca272f7bac76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:21.183702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 403/807] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:21.184559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad06871-08fd-4804-9f3f-8b74949d3222 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:21.262153 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:21.276970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 405/808] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:21.276970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d71e6f-b66b-442e-9d00-ca272f7bac76 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:21.321208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad06871-08fd-4804-9f3f-8b74949d3222 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:21.359169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766f26e-f8aa-40dd-b428-ba57a1ec7fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:21.360121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766f26e-f8aa-40dd-b428-ba57a1ec7fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:21.360394 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766f26e-f8aa-40dd-b428-ba57a1ec7fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:21.360713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766f26e-f8aa-40dd-b428-ba57a1ec7fb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:21.376568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766f26e-f8aa-40dd-b428-ba57a1ec7fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:21.385890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766f26e-f8aa-40dd-b428-ba57a1ec7fb9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-0766f26e-f8aa-40dd-b428-ba57a1ec7fb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:21.397979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:21.399062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:21.399954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:21.399954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:21.399954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:21.404319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 404/809] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:21.405398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766f26e-f8aa-40dd-b428-ba57a1ec7fb9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:21.408514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:21.423370 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d71e6f-b66b-442e-9d00-ca272f7bac76 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:21.467672 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbac58-1391-413a-9dab-e0600e805776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:21.469502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbac58-1391-413a-9dab-e0600e805776 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:21.469502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbac58-1391-413a-9dab-e0600e805776 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:21.469502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbac58-1391-413a-9dab-e0600e805776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:21.521318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766f26e-f8aa-40dd-b428-ba57a1ec7fb9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:21.548538 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:21.556107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbac58-1391-413a-9dab-e0600e805776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:21.557585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbac58-1391-413a-9dab-e0600e805776 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:21.563868 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a220d-b4a2-43c7-b3eb-fdc0ebe715ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:21.564471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a220d-b4a2-43c7-b3eb-fdc0ebe715ba None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:21.568762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a220d-b4a2-43c7-b3eb-fdc0ebe715ba None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:21.568762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a220d-b4a2-43c7-b3eb-fdc0ebe715ba None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/88fe81f7b4c04e9f9a26b430b39ccd36/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:21.568762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a220d-b4a2-43c7-b3eb-fdc0ebe715ba None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=88fe81f7b4c04e9f9a26b430b39ccd36, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:21.568762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a220d-b4a2-43c7-b3eb-fdc0ebe715ba None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:21.666954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 406/810] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:58:21.667912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbac58-1391-413a-9dab-e0600e805776 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:21.712392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 405/811] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:58:21 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/88fe81f7b4c04e9f9a26b430b39ccd36/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:58:21.713405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a220d-b4a2-43c7-b3eb-fdc0ebe715ba None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:21.788678 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:21.793298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:21.795761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbac58-1391-413a-9dab-e0600e805776 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:21.833182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca7614-be68-4b4f-8394-454c7e42414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:21.833911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca7614-be68-4b4f-8394-454c7e42414b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:21.834089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca7614-be68-4b4f-8394-454c7e42414b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:21.834355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca7614-be68-4b4f-8394-454c7e42414b None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:21.835014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca7614-be68-4b4f-8394-454c7e42414b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:21.835324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca7614-be68-4b4f-8394-454c7e42414b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:21.836750 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca7614-be68-4b4f-8394-454c7e42414b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:21.837919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a220d-b4a2-43c7-b3eb-fdc0ebe715ba None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:21.856584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 407/812] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:21.856584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca7614-be68-4b4f-8394-454c7e42414b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:21.886728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266578a-6a7a-48be-bb13-7f8136080bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:21.887442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266578a-6a7a-48be-bb13-7f8136080bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:21.887731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266578a-6a7a-48be-bb13-7f8136080bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:21.889018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266578a-6a7a-48be-bb13-7f8136080bd9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:21.889018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266578a-6a7a-48be-bb13-7f8136080bd9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:21.890139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266578a-6a7a-48be-bb13-7f8136080bd9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-8266578a-6a7a-48be-bb13-7f8136080bd9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:21.900101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:21.901482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:21.902502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 406/813] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:21 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:21.903490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266578a-6a7a-48be-bb13-7f8136080bd9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:21.981923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca7614-be68-4b4f-8394-454c7e42414b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.027055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dd435-f44d-4aa3-a6b9-ba429efda9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:22.028720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd435-f44d-4aa3-a6b9-ba429efda9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:22.029147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd435-f44d-4aa3-a6b9-ba429efda9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:22.029552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd435-f44d-4aa3-a6b9-ba429efda9f5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:22.030323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dd435-f44d-4aa3-a6b9-ba429efda9f5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:22.032354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266578a-6a7a-48be-bb13-7f8136080bd9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.032508 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dd435-f44d-4aa3-a6b9-ba429efda9f5 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-dd5dd435-f44d-4aa3-a6b9-ba429efda9f5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:22.045086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:22.046195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:22.047210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 408/814] 10.222.0.30 () {56 vars in 1129 bytes} [Wed Aug 24 09:58:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:22.048085 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dd435-f44d-4aa3-a6b9-ba429efda9f5 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:22.074756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:22.075375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:22.091069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0674b9-5be7-4ae4-bf53-8096da42197c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:22.092552 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0674b9-5be7-4ae4-bf53-8096da42197c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:22.092838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0674b9-5be7-4ae4-bf53-8096da42197c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:22.093143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0674b9-5be7-4ae4-bf53-8096da42197c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:22.105805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0674b9-5be7-4ae4-bf53-8096da42197c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:22.107440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0674b9-5be7-4ae4-bf53-8096da42197c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-df0674b9-5be7-4ae4-bf53-8096da42197c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:22.122833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:22.123859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:22.129833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:22.129833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:22.129833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:22.129833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 407/815] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:22.129833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0674b9-5be7-4ae4-bf53-8096da42197c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:22.167818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dd435-f44d-4aa3-a6b9-ba429efda9f5 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.208661 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec35e6-d5f3-4c5f-84fc-8e94700d41b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:22.209322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec35e6-d5f3-4c5f-84fc-8e94700d41b1 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:22.209602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec35e6-d5f3-4c5f-84fc-8e94700d41b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:22.209879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec35e6-d5f3-4c5f-84fc-8e94700d41b1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:22.218369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec35e6-d5f3-4c5f-84fc-8e94700d41b1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:22.219884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec35e6-d5f3-4c5f-84fc-8e94700d41b1 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-69ec35e6-d5f3-4c5f-84fc-8e94700d41b1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:22.236995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:22.244481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 409/816] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Aug 24 09:58:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:22.245795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec35e6-d5f3-4c5f-84fc-8e94700d41b1 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:22.248801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0674b9-5be7-4ae4-bf53-8096da42197c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.297465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0b7f4-9cc4-423d-bf27-4f5d353427cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:22.297465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0b7f4-9cc4-423d-bf27-4f5d353427cc None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:22.297465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0b7f4-9cc4-423d-bf27-4f5d353427cc None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:22.297465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0b7f4-9cc4-423d-bf27-4f5d353427cc None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:22.298115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0b7f4-9cc4-423d-bf27-4f5d353427cc None admin] RBAC: Adding query filter params (name=network) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:22.298115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0b7f4-9cc4-423d-bf27-4f5d353427cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:22.299109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0b7f4-9cc4-423d-bf27-4f5d353427cc None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:22.311509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 408/817] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:22.311509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0b7f4-9cc4-423d-bf27-4f5d353427cc None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:22.352099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:22.367013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec35e6-d5f3-4c5f-84fc-8e94700d41b1 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.402087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680e53b-286e-46b2-a902-f3704b42211e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:22.402854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680e53b-286e-46b2-a902-f3704b42211e None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:22.403140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680e53b-286e-46b2-a902-f3704b42211e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:22.403418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680e53b-286e-46b2-a902-f3704b42211e None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:22.404046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680e53b-286e-46b2-a902-f3704b42211e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:22.404355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680e53b-286e-46b2-a902-f3704b42211e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:22.405760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680e53b-286e-46b2-a902-f3704b42211e None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:22.416019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 410/818] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Aug 24 09:58:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:22.416019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680e53b-286e-46b2-a902-f3704b42211e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:22.420353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0b7f4-9cc4-423d-bf27-4f5d353427cc None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.434786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:22.458820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d66c-317a-4002-aaa4-4f7e9a994a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:22.459858 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d66c-317a-4002-aaa4-4f7e9a994a29 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:22.460305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d66c-317a-4002-aaa4-4f7e9a994a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:22.460749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d66c-317a-4002-aaa4-4f7e9a994a29 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:22.461442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3d66c-317a-4002-aaa4-4f7e9a994a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:22.461745 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3d66c-317a-4002-aaa4-4f7e9a994a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:22.463120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3d66c-317a-4002-aaa4-4f7e9a994a29 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:22.480947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 409/819] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:22.480947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d66c-317a-4002-aaa4-4f7e9a994a29 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:22.521608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680e53b-286e-46b2-a902-f3704b42211e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.556930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725154e-aecc-40e7-9e4e-68be6bd1aeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:22.557462 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725154e-aecc-40e7-9e4e-68be6bd1aeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:22.557746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725154e-aecc-40e7-9e4e-68be6bd1aeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:22.558021 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725154e-aecc-40e7-9e4e-68be6bd1aeb8 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:22.558750 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725154e-aecc-40e7-9e4e-68be6bd1aeb8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:22.559058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725154e-aecc-40e7-9e4e-68be6bd1aeb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:22.560482 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725154e-aecc-40e7-9e4e-68be6bd1aeb8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:22.579322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 411/820] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Aug 24 09:58:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:22.580050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725154e-aecc-40e7-9e4e-68be6bd1aeb8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:22.584312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:22.592931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d66c-317a-4002-aaa4-4f7e9a994a29 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.598601 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:22.629740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1a86-2ea7-436f-95f4-a74e4b871f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:22.630527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1a86-2ea7-436f-95f4-a74e4b871f9e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:22.630886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1a86-2ea7-436f-95f4-a74e4b871f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:22.631234 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1a86-2ea7-436f-95f4-a74e4b871f9e None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:22.631961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d1a86-2ea7-436f-95f4-a74e4b871f9e None admin] RBAC: Adding query filter params (type=network) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:22.632354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d1a86-2ea7-436f-95f4-a74e4b871f9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:22.633829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d1a86-2ea7-436f-95f4-a74e4b871f9e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:22.645241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 410/821] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:22.645241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1a86-2ea7-436f-95f4-a74e4b871f9e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:22.755218 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1a86-2ea7-436f-95f4-a74e4b871f9e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.795807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f308f0-7957-47db-a14d-8c5816146966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:22.796444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f308f0-7957-47db-a14d-8c5816146966 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:22.796725 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f308f0-7957-47db-a14d-8c5816146966 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:22.797029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f308f0-7957-47db-a14d-8c5816146966 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:22.804479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f308f0-7957-47db-a14d-8c5816146966 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:22.805961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f308f0-7957-47db-a14d-8c5816146966 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:22.807700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725154e-aecc-40e7-9e4e-68be6bd1aeb8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-c0f308f0-7957-47db-a14d-8c5816146966 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:22.817770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:22.818984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:22.819898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:22.819898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:22.819898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:22.819898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:22.819898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:22.819898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:22.819898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:22.819898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:22.819898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 411/822] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:22.820427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f308f0-7957-47db-a14d-8c5816146966 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:22.851579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b7806-35b6-4f37-8bd0-5ac0df74206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:22.852199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b7806-35b6-4f37-8bd0-5ac0df74206a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:22.852476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b7806-35b6-4f37-8bd0-5ac0df74206a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:22.852745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b7806-35b6-4f37-8bd0-5ac0df74206a None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:22.853423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b7806-35b6-4f37-8bd0-5ac0df74206a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:22.853571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:22.853740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b7806-35b6-4f37-8bd0-5ac0df74206a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:22.855146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b7806-35b6-4f37-8bd0-5ac0df74206a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:22.860611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 412/823] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Aug 24 09:58:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:22.866108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b7806-35b6-4f37-8bd0-5ac0df74206a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:22.939164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:22.942056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f308f0-7957-47db-a14d-8c5816146966 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.978623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b7806-35b6-4f37-8bd0-5ac0df74206a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:22.979180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72464006-0ac7-4199-a3d4-ce18c633a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:22.979816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72464006-0ac7-4199-a3d4-ce18c633a6bb None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:22.980423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72464006-0ac7-4199-a3d4-ce18c633a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:22.980700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72464006-0ac7-4199-a3d4-ce18c633a6bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:22.991930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72464006-0ac7-4199-a3d4-ce18c633a6bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:22.993412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72464006-0ac7-4199-a3d4-ce18c633a6bb None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-72464006-0ac7-4199-a3d4-ce18c633a6bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:23.009847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:23.010843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:23.011769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:23.011769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:23.011769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:23.011769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 412/824] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:23.012003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72464006-0ac7-4199-a3d4-ce18c633a6bb None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:23.021624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f540634-e917-4154-80bd-dad456d4ca3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:23.022257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f540634-e917-4154-80bd-dad456d4ca3f None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:23.022534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f540634-e917-4154-80bd-dad456d4ca3f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:23.022801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f540634-e917-4154-80bd-dad456d4ca3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:23.023451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f540634-e917-4154-80bd-dad456d4ca3f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:23.024802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f540634-e917-4154-80bd-dad456d4ca3f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:23.092452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:23.112950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72464006-0ac7-4199-a3d4-ce18c633a6bb None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:23.147013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425e232-eca2-4ddd-b0fc-dd5ed69fb6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:23.147608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425e232-eca2-4ddd-b0fc-dd5ed69fb6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:23.147947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425e232-eca2-4ddd-b0fc-dd5ed69fb6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:23.148217 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425e232-eca2-4ddd-b0fc-dd5ed69fb6f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:23.148952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425e232-eca2-4ddd-b0fc-dd5ed69fb6f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:23.149251 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425e232-eca2-4ddd-b0fc-dd5ed69fb6f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:23.150657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425e232-eca2-4ddd-b0fc-dd5ed69fb6f8 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:23.162925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 413/825] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:23.163715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425e232-eca2-4ddd-b0fc-dd5ed69fb6f8 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:23.193516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 413/826] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 09:58:22 2022] POST /identity/v3/endpoints => generated 337 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:23.194400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f540634-e917-4154-80bd-dad456d4ca3f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:23.306890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425e232-eca2-4ddd-b0fc-dd5ed69fb6f8 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:23.359764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f540634-e917-4154-80bd-dad456d4ca3f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:23.360853 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1e970-3aa9-4ff9-a4d2-4db1645799b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:23.361918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e970-3aa9-4ff9-a4d2-4db1645799b9 None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:23.362206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e970-3aa9-4ff9-a4d2-4db1645799b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:23.363061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e970-3aa9-4ff9-a4d2-4db1645799b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:23.363733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1e970-3aa9-4ff9-a4d2-4db1645799b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:23.365207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1e970-3aa9-4ff9-a4d2-4db1645799b9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:23.372840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:23.384209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:23.403975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b79b2-5aad-4e24-b992-b4a1e5c2a7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:23.404588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b79b2-5aad-4e24-b992-b4a1e5c2a7af None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:23.404951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b79b2-5aad-4e24-b992-b4a1e5c2a7af None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:23.405233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b79b2-5aad-4e24-b992-b4a1e5c2a7af None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:23.405798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b79b2-5aad-4e24-b992-b4a1e5c2a7af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:23.406096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b79b2-5aad-4e24-b992-b4a1e5c2a7af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:23.407462 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b79b2-5aad-4e24-b992-b4a1e5c2a7af None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:23.422545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 414/827] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:23.423715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b79b2-5aad-4e24-b992-b4a1e5c2a7af None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:23.496982 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 414/828] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 09:58:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:23.498534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1e970-3aa9-4ff9-a4d2-4db1645799b9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:23.534046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b79b2-5aad-4e24-b992-b4a1e5c2a7af None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:23.580705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed2ad7-34e2-45a6-9fe8-278e0605acb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:23.581394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed2ad7-34e2-45a6-9fe8-278e0605acb7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:23.581731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed2ad7-34e2-45a6-9fe8-278e0605acb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:23.582096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed2ad7-34e2-45a6-9fe8-278e0605acb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:23.583134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed2ad7-34e2-45a6-9fe8-278e0605acb7 None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=7cd6d2291f934fbdb25c410c2c3abcb6) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:23.583430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed2ad7-34e2-45a6-9fe8-278e0605acb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:23.585058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed2ad7-34e2-45a6-9fe8-278e0605acb7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:23.597639 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 415/829] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:23 2022] GET /identity/v3/role_assignments?user.id=7cd6d2291f934fbdb25c410c2c3abcb6&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:23.636837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1e970-3aa9-4ff9-a4d2-4db1645799b9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:23.678445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84349ba9-53cf-48dd-9142-b95dca5b6914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:23.679503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84349ba9-53cf-48dd-9142-b95dca5b6914 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:23.679994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84349ba9-53cf-48dd-9142-b95dca5b6914 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:23.680467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84349ba9-53cf-48dd-9142-b95dca5b6914 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:23.688702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84349ba9-53cf-48dd-9142-b95dca5b6914 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:23.691144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84349ba9-53cf-48dd-9142-b95dca5b6914 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-84349ba9-53cf-48dd-9142-b95dca5b6914 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:23.704094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:23.705202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:23.706310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:23.706310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:23.706310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:23.706310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:23.706310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:23.706310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:23.706310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:23.706310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:23.706744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 415/830] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:23.715913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed2ad7-34e2-45a6-9fe8-278e0605acb7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:23.734375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d360c-0899-4459-bfb7-b1d8c61dffc5 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:23.734717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d360c-0899-4459-bfb7-b1d8c61dffc5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:23.734932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d360c-0899-4459-bfb7-b1d8c61dffc5 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:23.736638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 416/831] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:23.742881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1832093-e807-46ee-a490-34bec3fd3784 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:23.743196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1832093-e807-46ee-a490-34bec3fd3784 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:23.743426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1832093-e807-46ee-a490-34bec3fd3784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:23.766680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-a1832093-e807-46ee-a490-34bec3fd3784 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:23.768237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a1832093-e807-46ee-a490-34bec3fd3784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:23.770204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:23.826925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:23.846596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed2ad7-34e2-45a6-9fe8-278e0605acb7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:23.852028 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 417/832] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:23.887895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424ec05-e473-44c3-8dee-cc388d0f7856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:23.888435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424ec05-e473-44c3-8dee-cc388d0f7856 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:23.888763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424ec05-e473-44c3-8dee-cc388d0f7856 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:23.889065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424ec05-e473-44c3-8dee-cc388d0f7856 None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:23.889804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424ec05-e473-44c3-8dee-cc388d0f7856 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:23.890102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424ec05-e473-44c3-8dee-cc388d0f7856 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:23.891518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424ec05-e473-44c3-8dee-cc388d0f7856 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:23.905990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 416/833] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:23.912644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1832093-e807-46ee-a490-34bec3fd3784 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:24.023794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1832093-e807-46ee-a490-34bec3fd3784 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:24.063835 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b2737-4f3e-43f5-bb79-e678694d96fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:24.064919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b2737-4f3e-43f5-bb79-e678694d96fe None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:24.065543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b2737-4f3e-43f5-bb79-e678694d96fe None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:24.066016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b2737-4f3e-43f5-bb79-e678694d96fe None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/8b7227998e7846c7a9bc14b12b637ee0/roles/f9446582c6a34010bd3ae841eba530bc` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:24.083534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b2737-4f3e-43f5-bb79-e678694d96fe None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=8b7227998e7846c7a9bc14b12b637ee0, role_id=f9446582c6a34010bd3ae841eba530bc)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:24.085425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b2737-4f3e-43f5-bb79-e678694d96fe None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:24.140144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424ec05-e473-44c3-8dee-cc388d0f7856 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:24.164218 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 418/834] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:58:23 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/8b7227998e7846c7a9bc14b12b637ee0/roles/f9446582c6a34010bd3ae841eba530bc => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:58:24.278296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424ec05-e473-44c3-8dee-cc388d0f7856 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:24.281223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:24.323270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d433a-6681-4056-8cb6-c6aaf4c47ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:24.323897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d433a-6681-4056-8cb6-c6aaf4c47ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:24.324172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d433a-6681-4056-8cb6-c6aaf4c47ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:24.324440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d433a-6681-4056-8cb6-c6aaf4c47ea7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:24.336346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d433a-6681-4056-8cb6-c6aaf4c47ea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-3c5d433a-6681-4056-8cb6-c6aaf4c47ea7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:24.341446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:24.342621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:24.343665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:24.346451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 417/835] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:24.353009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b2737-4f3e-43f5-bb79-e678694d96fe None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:24.387156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:24.467801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b2737-4f3e-43f5-bb79-e678694d96fe None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:24.502004 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2afed4-da38-45af-b270-b16eebb6a902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:24.502612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2afed4-da38-45af-b270-b16eebb6a902 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:24.502930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2afed4-da38-45af-b270-b16eebb6a902 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:24.503211 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2afed4-da38-45af-b270-b16eebb6a902 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:24.503861 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2afed4-da38-45af-b270-b16eebb6a902 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:24.504202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2afed4-da38-45af-b270-b16eebb6a902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:24.505893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2afed4-da38-45af-b270-b16eebb6a902 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:24.515434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 419/836] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:58:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:24.521722 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d433a-6681-4056-8cb6-c6aaf4c47ea7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:24.679527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d433a-6681-4056-8cb6-c6aaf4c47ea7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:24.717165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6844f-4f8d-4c28-8f01-728bcc46c3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:24.717858 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6844f-4f8d-4c28-8f01-728bcc46c3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:24.718084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6844f-4f8d-4c28-8f01-728bcc46c3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:24.718358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6844f-4f8d-4c28-8f01-728bcc46c3c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:24.730383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6844f-4f8d-4c28-8f01-728bcc46c3c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:24.731845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6844f-4f8d-4c28-8f01-728bcc46c3c8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-70d6844f-4f8d-4c28-8f01-728bcc46c3c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:24.747724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:24.748732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:24.749695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:24.749695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:24.749695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:24.749695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 418/837] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:24.755479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2afed4-da38-45af-b270-b16eebb6a902 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:24.859250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:24.869487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2afed4-da38-45af-b270-b16eebb6a902 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:24.908225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8909d-44dc-44cd-8791-3e8b1e956556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:24.908833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8909d-44dc-44cd-8791-3e8b1e956556 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:24.909791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8909d-44dc-44cd-8791-3e8b1e956556 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:24.910098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8909d-44dc-44cd-8791-3e8b1e956556 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:24.910746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8909d-44dc-44cd-8791-3e8b1e956556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:24.911306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8909d-44dc-44cd-8791-3e8b1e956556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:24.912675 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8909d-44dc-44cd-8791-3e8b1e956556 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:24.930416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 420/838] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:24.932579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6844f-4f8d-4c28-8f01-728bcc46c3c8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:24.947490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:25.040026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6844f-4f8d-4c28-8f01-728bcc46c3c8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:25.075112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240218a3-1b12-4826-9e94-bf9208f233bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:25.075781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240218a3-1b12-4826-9e94-bf9208f233bc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:25.076070 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240218a3-1b12-4826-9e94-bf9208f233bc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:25.076342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240218a3-1b12-4826-9e94-bf9208f233bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:25.084191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240218a3-1b12-4826-9e94-bf9208f233bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:25.085791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240218a3-1b12-4826-9e94-bf9208f233bc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-240218a3-1b12-4826-9e94-bf9208f233bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:25.098377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:25.101161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:25.102379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:25.102379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:25.102379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:25.102379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:25.102379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:25.102379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:25.102379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:25.102379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:25.102379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:58:25.102379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:25.102379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 419/839] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:25.106116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8909d-44dc-44cd-8791-3e8b1e956556 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:25.227392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8909d-44dc-44cd-8791-3e8b1e956556 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:25.267790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85022b-96cd-4d40-926e-7e2c27c2fece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:25.268413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85022b-96cd-4d40-926e-7e2c27c2fece None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:25.269236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85022b-96cd-4d40-926e-7e2c27c2fece None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:25.269512 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85022b-96cd-4d40-926e-7e2c27c2fece None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:25.270255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85022b-96cd-4d40-926e-7e2c27c2fece None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:25.270562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85022b-96cd-4d40-926e-7e2c27c2fece None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:25.271993 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85022b-96cd-4d40-926e-7e2c27c2fece None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:25.297176 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 421/840] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Aug 24 09:58:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:25.301985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240218a3-1b12-4826-9e94-bf9208f233bc None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:25.376299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:25.410456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240218a3-1b12-4826-9e94-bf9208f233bc None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:25.449741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a4f98-fb22-4eb5-bc9d-3dbcab0ec847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:25.450351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4f98-fb22-4eb5-bc9d-3dbcab0ec847 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:25.450631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4f98-fb22-4eb5-bc9d-3dbcab0ec847 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:25.450907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4f98-fb22-4eb5-bc9d-3dbcab0ec847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:25.521906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a4f98-fb22-4eb5-bc9d-3dbcab0ec847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:25.523375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a4f98-fb22-4eb5-bc9d-3dbcab0ec847 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:25.600640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:25.626762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 420/841] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:25 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:58:25.633297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85022b-96cd-4d40-926e-7e2c27c2fece None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:25.659884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:25.683533 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb83a7c-3de4-4d06-ad33-5cf503712011 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:25.684203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb83a7c-3de4-4d06-ad33-5cf503712011 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:25.684203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb83a7c-3de4-4d06-ad33-5cf503712011 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:25.686160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 421/842] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:58:25.691366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca3d0-bf95-4e6a-aa75-5d12df1c278a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:25.691910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca3d0-bf95-4e6a-aa75-5d12df1c278a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:25.692385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca3d0-bf95-4e6a-aa75-5d12df1c278a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:25.712927 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-317ca3d0-bf95-4e6a-aa75-5d12df1c278a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:25.714462 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-317ca3d0-bf95-4e6a-aa75-5d12df1c278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:25.748051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85022b-96cd-4d40-926e-7e2c27c2fece None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:25.786423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e5339-c189-4ad0-97d0-72d54a93811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:25.787433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e5339-c189-4ad0-97d0-72d54a93811e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:25.787948 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e5339-c189-4ad0-97d0-72d54a93811e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:25.788282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e5339-c189-4ad0-97d0-72d54a93811e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:25.797952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 422/843] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:25.799213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e5339-c189-4ad0-97d0-72d54a93811e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:25.801844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e5339-c189-4ad0-97d0-72d54a93811e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b10e5339-c189-4ad0-97d0-72d54a93811e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:25.816729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:25.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:25.819383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:25.819383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:25.819383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:25.819383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 422/844] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:25.823663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ca3d0-bf95-4e6a-aa75-5d12df1c278a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:25.853353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a025b-ca44-4e2b-92b7-18681360211c None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:25.853551 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a025b-ca44-4e2b-92b7-18681360211c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:25.853808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a025b-ca44-4e2b-92b7-18681360211c None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:25.855442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 423/845] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:58:25.862028 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a61850-db55-4f47-9b85-fa8539ce4f5c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:25.862699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a61850-db55-4f47-9b85-fa8539ce4f5c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:25.863099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a61850-db55-4f47-9b85-fa8539ce4f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:25.886075 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-98a61850-db55-4f47-9b85-fa8539ce4f5c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:25.888025 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-98a61850-db55-4f47-9b85-fa8539ce4f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:25.937232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:25.940885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ca3d0-bf95-4e6a-aa75-5d12df1c278a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:25.964505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:25.976023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd38d3-1dc2-43a9-93cd-881ef77d1913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:25.976637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd38d3-1dc2-43a9-93cd-881ef77d1913 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:25.976939 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd38d3-1dc2-43a9-93cd-881ef77d1913 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:25.977213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd38d3-1dc2-43a9-93cd-881ef77d1913 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:25.977919 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd38d3-1dc2-43a9-93cd-881ef77d1913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:25.978221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd38d3-1dc2-43a9-93cd-881ef77d1913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:25.979409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 424/846] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:25.985598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd38d3-1dc2-43a9-93cd-881ef77d1913 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:26.001571 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 423/847] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:26.007862 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a61850-db55-4f47-9b85-fa8539ce4f5c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:26.064079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd38d3-1dc2-43a9-93cd-881ef77d1913 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:26.139260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a61850-db55-4f47-9b85-fa8539ce4f5c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:26.173053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd38d3-1dc2-43a9-93cd-881ef77d1913 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:26.178771 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96030f9-d8e6-443c-9c92-2791d2cf5492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:26.179410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96030f9-d8e6-443c-9c92-2791d2cf5492 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:26.179682 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96030f9-d8e6-443c-9c92-2791d2cf5492 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:26.180015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96030f9-d8e6-443c-9c92-2791d2cf5492 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:26.187600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96030f9-d8e6-443c-9c92-2791d2cf5492 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:26.189062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96030f9-d8e6-443c-9c92-2791d2cf5492 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-d96030f9-d8e6-443c-9c92-2791d2cf5492 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:26.202114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:26.203067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:26.203965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:26.203965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:26.203965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:26.203965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:26.203965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:26.203965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:26.203965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:26.203965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:26.203965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 425/848] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:26.209836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96030f9-d8e6-443c-9c92-2791d2cf5492 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:26.212379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b8426-0923-48f1-a2a0-03c14dd9af8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:26.213111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8426-0923-48f1-a2a0-03c14dd9af8c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:26.213460 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8426-0923-48f1-a2a0-03c14dd9af8c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:26.213801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8426-0923-48f1-a2a0-03c14dd9af8c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:26.214456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b8426-0923-48f1-a2a0-03c14dd9af8c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:26.215881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b8426-0923-48f1-a2a0-03c14dd9af8c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-238b8426-0923-48f1-a2a0-03c14dd9af8c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:26.225424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:26.227619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:58:26.229633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:26.236942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 424/849] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:58:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:26.236942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b8426-0923-48f1-a2a0-03c14dd9af8c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:26.261941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:26.272701 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:26.324137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96030f9-d8e6-443c-9c92-2791d2cf5492 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:26.349304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b8426-0923-48f1-a2a0-03c14dd9af8c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:26.358414 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234672f-a7cd-46f4-9f60-a3e27523bcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:26.359366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234672f-a7cd-46f4-9f60-a3e27523bcc9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:26.359645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234672f-a7cd-46f4-9f60-a3e27523bcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:26.359938 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234672f-a7cd-46f4-9f60-a3e27523bcc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:26.360609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234672f-a7cd-46f4-9f60-a3e27523bcc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:26.360944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234672f-a7cd-46f4-9f60-a3e27523bcc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:26.362924 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234672f-a7cd-46f4-9f60-a3e27523bcc9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:26.375421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 426/850] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:26.376231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234672f-a7cd-46f4-9f60-a3e27523bcc9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:26.389204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be5c78-f1c8-41f4-87dd-7bcc58caf95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:26.389888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be5c78-f1c8-41f4-87dd-7bcc58caf95a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:26.390148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be5c78-f1c8-41f4-87dd-7bcc58caf95a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:26.390374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be5c78-f1c8-41f4-87dd-7bcc58caf95a None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:26.391000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be5c78-f1c8-41f4-87dd-7bcc58caf95a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:26.391301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be5c78-f1c8-41f4-87dd-7bcc58caf95a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:26.392660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be5c78-f1c8-41f4-87dd-7bcc58caf95a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:26.397513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 425/851] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:58:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:26.398863 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f458da-b29b-45da-bacf-2faa596ae9bc None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:26.399309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f458da-b29b-45da-bacf-2faa596ae9bc None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:26.399746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f458da-b29b-45da-bacf-2faa596ae9bc None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:26.401119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 426/852] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:26.402227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f458da-b29b-45da-bacf-2faa596ae9bc None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:26.481644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234672f-a7cd-46f4-9f60-a3e27523bcc9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:26.494810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:26.499181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:26.512767 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f458da-b29b-45da-bacf-2faa596ae9bc None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:26.515809 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f8f8e-83eb-4acf-9819-60c2af9a7408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:26.516441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f8f8e-83eb-4acf-9819-60c2af9a7408 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:26.516721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f8f8e-83eb-4acf-9819-60c2af9a7408 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:26.517036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f8f8e-83eb-4acf-9819-60c2af9a7408 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:26.517620 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f8f8e-83eb-4acf-9819-60c2af9a7408 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:26.519716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f8f8e-83eb-4acf-9819-60c2af9a7408 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-765f8f8e-83eb-4acf-9819-60c2af9a7408 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:26.528477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:26.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:26.530360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 427/853] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Aug 24 09:58:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:26.531506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f8f8e-83eb-4acf-9819-60c2af9a7408 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:26.550532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf695cda-df36-41c8-8e73-b70fc0baccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:26.550956 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf695cda-df36-41c8-8e73-b70fc0baccb6 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:26.555813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf695cda-df36-41c8-8e73-b70fc0baccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:26.555813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf695cda-df36-41c8-8e73-b70fc0baccb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:26.555813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf695cda-df36-41c8-8e73-b70fc0baccb6 None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=88fe81f7b4c04e9f9a26b430b39ccd36) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:26.555813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf695cda-df36-41c8-8e73-b70fc0baccb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:26.555813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf695cda-df36-41c8-8e73-b70fc0baccb6 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:26.559730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 427/854] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:26 2022] GET /identity/v3/role_assignments?user.id=88fe81f7b4c04e9f9a26b430b39ccd36&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:26.560931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c0775-9693-44b2-a6c9-f463ff718b13 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:26.561220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c0775-9693-44b2-a6c9-f463ff718b13 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:26.561498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c0775-9693-44b2-a6c9-f463ff718b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:26.583478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-b50c0775-9693-44b2-a6c9-f463ff718b13 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:26.585761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b50c0775-9693-44b2-a6c9-f463ff718b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:26.649872 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f8f8e-83eb-4acf-9819-60c2af9a7408 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:26.673099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 428/855] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:26.673099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c0775-9693-44b2-a6c9-f463ff718b13 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:26.704161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44497710-f937-42e5-83e4-1c4c6d233efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:26.705051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44497710-f937-42e5-83e4-1c4c6d233efb None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:26.705338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44497710-f937-42e5-83e4-1c4c6d233efb None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:26.705610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44497710-f937-42e5-83e4-1c4c6d233efb None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:26.706444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44497710-f937-42e5-83e4-1c4c6d233efb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:26.706748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44497710-f937-42e5-83e4-1c4c6d233efb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:26.708130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44497710-f937-42e5-83e4-1c4c6d233efb None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:26.713625 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 428/856] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:58:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:26.780834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:26.803326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c0775-9693-44b2-a6c9-f463ff718b13 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:26.842985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa4efd-1a0c-43ee-b6a1-374625edbb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:26.843623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa4efd-1a0c-43ee-b6a1-374625edbb99 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:26.843917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa4efd-1a0c-43ee-b6a1-374625edbb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:26.844195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa4efd-1a0c-43ee-b6a1-374625edbb99 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:26.844843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa4efd-1a0c-43ee-b6a1-374625edbb99 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:26.845180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa4efd-1a0c-43ee-b6a1-374625edbb99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:26.846747 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa4efd-1a0c-43ee-b6a1-374625edbb99 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:26.852757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 429/857] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Aug 24 09:58:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:26.858565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44497710-f937-42e5-83e4-1c4c6d233efb None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:26.908580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:26.947161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa4efd-1a0c-43ee-b6a1-374625edbb99 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:26.973679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44497710-f937-42e5-83e4-1c4c6d233efb None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:27.010627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deced6-9206-473d-a36e-fc777ce53d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:27.011227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deced6-9206-473d-a36e-fc777ce53d31 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:27.011503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deced6-9206-473d-a36e-fc777ce53d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:27.011771 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deced6-9206-473d-a36e-fc777ce53d31 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:27.012399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9deced6-9206-473d-a36e-fc777ce53d31 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:27.012754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9deced6-9206-473d-a36e-fc777ce53d31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:27.014234 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9deced6-9206-473d-a36e-fc777ce53d31 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:27.020133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 429/858] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Aug 24 09:58:26 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:27.080754 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa4efd-1a0c-43ee-b6a1-374625edbb99 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:27.120641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc1472-f406-47f8-8731-b4524d1bf906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:27.121366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc1472-f406-47f8-8731-b4524d1bf906 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:27.121645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc1472-f406-47f8-8731-b4524d1bf906 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:27.121918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc1472-f406-47f8-8731-b4524d1bf906 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:27.137126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc1472-f406-47f8-8731-b4524d1bf906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-cacc1472-f406-47f8-8731-b4524d1bf906 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:27.142715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:27.144382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:27.145518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 430/859] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 09:58:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:27.154501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deced6-9206-473d-a36e-fc777ce53d31 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:27.269818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deced6-9206-473d-a36e-fc777ce53d31 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:27.273555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:27.313833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d2c62-6cd6-4631-97a4-5b8f7a16dd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:27.314765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2c62-6cd6-4631-97a4-5b8f7a16dd4e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:27.315761 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2c62-6cd6-4631-97a4-5b8f7a16dd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:27.316219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2c62-6cd6-4631-97a4-5b8f7a16dd4e None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:27.317123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d2c62-6cd6-4631-97a4-5b8f7a16dd4e None admin] RBAC: Adding query filter params (name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:27.317609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d2c62-6cd6-4631-97a4-5b8f7a16dd4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:27.319323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d2c62-6cd6-4631-97a4-5b8f7a16dd4e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:27.328576 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 430/860] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 09:58:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:27.334847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc1472-f406-47f8-8731-b4524d1bf906 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:27.344447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:27.446120 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc1472-f406-47f8-8731-b4524d1bf906 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:27.486236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514a8d6-9664-48a3-9d8f-0407fa7b7948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:27.486919 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514a8d6-9664-48a3-9d8f-0407fa7b7948 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:27.487237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514a8d6-9664-48a3-9d8f-0407fa7b7948 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:27.487671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514a8d6-9664-48a3-9d8f-0407fa7b7948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:27.499244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514a8d6-9664-48a3-9d8f-0407fa7b7948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:27.500764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514a8d6-9664-48a3-9d8f-0407fa7b7948 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-b514a8d6-9664-48a3-9d8f-0407fa7b7948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:27.527368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:27.529216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:27.530870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:27.530870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:27.530870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:27.530870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 431/861] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:27.535498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d2c62-6cd6-4631-97a4-5b8f7a16dd4e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:27.693385 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d2c62-6cd6-4631-97a4-5b8f7a16dd4e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:27.752747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c2f5e-6735-49a6-ac8e-4e6ade156c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:27.754891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c2f5e-6735-49a6-ac8e-4e6ade156c0f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:27.755571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c2f5e-6735-49a6-ac8e-4e6ade156c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:27.756223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c2f5e-6735-49a6-ac8e-4e6ade156c0f None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:27.757489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c2f5e-6735-49a6-ac8e-4e6ade156c0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:27.758205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c2f5e-6735-49a6-ac8e-4e6ade156c0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:27.760718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c2f5e-6735-49a6-ac8e-4e6ade156c0f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:27.783048 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 431/862] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:27.789016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514a8d6-9664-48a3-9d8f-0407fa7b7948 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:27.849892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:27.894573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514a8d6-9664-48a3-9d8f-0407fa7b7948 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:27.930740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc533333-4c41-427a-9b7c-5c81d5fcb7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:27.931367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc533333-4c41-427a-9b7c-5c81d5fcb7ea None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:27.931645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc533333-4c41-427a-9b7c-5c81d5fcb7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:27.931915 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc533333-4c41-427a-9b7c-5c81d5fcb7ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:27.939892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc533333-4c41-427a-9b7c-5c81d5fcb7ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:27.941399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc533333-4c41-427a-9b7c-5c81d5fcb7ea None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-bc533333-4c41-427a-9b7c-5c81d5fcb7ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:27.952765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:27.953715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:27.954662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:27.954662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:27.954662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:27.954662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:27.954662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:27.954662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:27.954662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:27.954662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:27.954662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:27.954662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:27.954662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 432/863] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Aug 24 09:58:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:27.959918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c2f5e-6735-49a6-ac8e-4e6ade156c0f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:27.989446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:28.072435 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c2f5e-6735-49a6-ac8e-4e6ade156c0f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:28.107840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50b9f4-c257-484d-8ce7-5bcb00af0f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:28.108441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50b9f4-c257-484d-8ce7-5bcb00af0f31 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:28.108716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50b9f4-c257-484d-8ce7-5bcb00af0f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:28.109012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50b9f4-c257-484d-8ce7-5bcb00af0f31 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:28.109740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50b9f4-c257-484d-8ce7-5bcb00af0f31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:28.110039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50b9f4-c257-484d-8ce7-5bcb00af0f31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:28.111525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50b9f4-c257-484d-8ce7-5bcb00af0f31 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:28.130837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 432/864] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Aug 24 09:58:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:28.138219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc533333-4c41-427a-9b7c-5c81d5fcb7ea None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:28.248654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc533333-4c41-427a-9b7c-5c81d5fcb7ea None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:28.272778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:28.302532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cfee7-15d4-4575-8129-8e3b6aa73aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:28.303155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cfee7-15d4-4575-8129-8e3b6aa73aba None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:28.303432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cfee7-15d4-4575-8129-8e3b6aa73aba None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:28.303705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cfee7-15d4-4575-8129-8e3b6aa73aba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:28.376243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cfee7-15d4-4575-8129-8e3b6aa73aba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:28.378342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cfee7-15d4-4575-8129-8e3b6aa73aba None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:28.481698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 433/865] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:58:28.487959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50b9f4-c257-484d-8ce7-5bcb00af0f31 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:28.583606 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:28.599823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50b9f4-c257-484d-8ce7-5bcb00af0f31 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:28.629655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b27cf-20b8-4bbc-aec5-e9243b1223f1 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:28.629961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b27cf-20b8-4bbc-aec5-e9243b1223f1 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:28.630129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b27cf-20b8-4bbc-aec5-e9243b1223f1 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:28.631724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 434/866] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:28.635942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b75871-3e51-4eb5-b640-d785cbf6f7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:28.636575 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b75871-3e51-4eb5-b640-d785cbf6f7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:28.636758 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887d20-4a0e-4a25-86eb-f6f6c3b290e8 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:28.636948 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b75871-3e51-4eb5-b640-d785cbf6f7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:28.637162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887d20-4a0e-4a25-86eb-f6f6c3b290e8 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:28.637343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b75871-3e51-4eb5-b640-d785cbf6f7a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:28.637439 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887d20-4a0e-4a25-86eb-f6f6c3b290e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:28.648525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b75871-3e51-4eb5-b640-d785cbf6f7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:28.650014 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b75871-3e51-4eb5-b640-d785cbf6f7a4 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:28.655730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-bd887d20-4a0e-4a25-86eb-f6f6c3b290e8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:28.656994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-bd887d20-4a0e-4a25-86eb-f6f6c3b290e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-26b75871-3e51-4eb5-b640-d785cbf6f7a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:28.666695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:28.667721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:28.668633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:28.668633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:28.668633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:28.668633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 433/867] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:28.672404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:28.677864 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b75871-3e51-4eb5-b640-d785cbf6f7a4 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:28.735019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 435/868] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:28.802484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b75871-3e51-4eb5-b640-d785cbf6f7a4 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:28.838180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eba641-97b1-4913-94d2-08106b4f1cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:28.838789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eba641-97b1-4913-94d2-08106b4f1cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:28.839063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eba641-97b1-4913-94d2-08106b4f1cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:28.839329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eba641-97b1-4913-94d2-08106b4f1cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:28.839967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eba641-97b1-4913-94d2-08106b4f1cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:28.840316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eba641-97b1-4913-94d2-08106b4f1cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:28.841761 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eba641-97b1-4913-94d2-08106b4f1cb5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:28.844247 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:28.855133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 434/869] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:28.860729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd887d20-4a0e-4a25-86eb-f6f6c3b290e8 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:28.950324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b92eab-0dff-4628-a1e4-3a5a1a05b2d5 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:28.950931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b92eab-0dff-4628-a1e4-3a5a1a05b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:28.951433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b92eab-0dff-4628-a1e4-3a5a1a05b2d5 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:28.952985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 435/870] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:58:28.962120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bb7dd-2fcf-4538-b40a-85455df25edb None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:28.962699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bb7dd-2fcf-4538-b40a-85455df25edb None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:28.963162 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bb7dd-2fcf-4538-b40a-85455df25edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:28.973700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd887d20-4a0e-4a25-86eb-f6f6c3b290e8 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:28.984833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-221bb7dd-2fcf-4538-b40a-85455df25edb None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:28.986510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-221bb7dd-2fcf-4538-b40a-85455df25edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:29.014953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064fcf5-d324-4e8f-8ec5-d2d0ac6879ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:29.016527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064fcf5-d324-4e8f-8ec5-d2d0ac6879ca None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:29.017063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064fcf5-d324-4e8f-8ec5-d2d0ac6879ca None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:29.017816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064fcf5-d324-4e8f-8ec5-d2d0ac6879ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:29.030345 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064fcf5-d324-4e8f-8ec5-d2d0ac6879ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:29.032416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064fcf5-d324-4e8f-8ec5-d2d0ac6879ca None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-4064fcf5-d324-4e8f-8ec5-d2d0ac6879ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:29.049151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:29.050782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:29.051791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:29.051791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:29.051791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:29.051791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:29.051791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:29.051791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:29.051791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:29.051791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:29.051791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 436/871] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:29.052272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064fcf5-d324-4e8f-8ec5-d2d0ac6879ca None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:29.066026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:29.074628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 436/872] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:29.080033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221bb7dd-2fcf-4538-b40a-85455df25edb None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:29.167991 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064fcf5-d324-4e8f-8ec5-d2d0ac6879ca None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:29.208535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221bb7dd-2fcf-4538-b40a-85455df25edb None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:29.208822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754230a-1d25-4862-9773-8a7ff302e759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:29.209672 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754230a-1d25-4862-9773-8a7ff302e759 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:29.210160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754230a-1d25-4862-9773-8a7ff302e759 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:29.210665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754230a-1d25-4862-9773-8a7ff302e759 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:29.211418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754230a-1d25-4862-9773-8a7ff302e759 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:29.212794 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754230a-1d25-4862-9773-8a7ff302e759 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-b754230a-1d25-4862-9773-8a7ff302e759 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:29.223333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:29.224362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:29.225668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 437/873] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Aug 24 09:58:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:29.226504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855533-f5cd-43e2-a1c5-d5d1c9f274bd None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:29.226504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855533-f5cd-43e2-a1c5-d5d1c9f274bd None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:29.226504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855533-f5cd-43e2-a1c5-d5d1c9f274bd None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:29.227707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 438/874] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:29.232783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18855533-f5cd-43e2-a1c5-d5d1c9f274bd None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:29.243003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:29.252288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd4882-3279-4a51-9da2-688f685523f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:29.252950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4882-3279-4a51-9da2-688f685523f2 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:29.257462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4882-3279-4a51-9da2-688f685523f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:29.257462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4882-3279-4a51-9da2-688f685523f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:29.257462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd4882-3279-4a51-9da2-688f685523f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:29.257462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd4882-3279-4a51-9da2-688f685523f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:29.257462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd4882-3279-4a51-9da2-688f685523f2 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:29.273950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 437/875] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:29.275182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1f617-8a21-49a8-ba64-da1e1ebf7c9a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:29.275522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1f617-8a21-49a8-ba64-da1e1ebf7c9a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:29.275851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1f617-8a21-49a8-ba64-da1e1ebf7c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:29.299628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-f1d1f617-8a21-49a8-ba64-da1e1ebf7c9a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:29.301224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f1d1f617-8a21-49a8-ba64-da1e1ebf7c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:29.322465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:29.358469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18855533-f5cd-43e2-a1c5-d5d1c9f274bd None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:29.371887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 438/876] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:29.372742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1f617-8a21-49a8-ba64-da1e1ebf7c9a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:29.396254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b50f9-9b4c-491c-9ea6-ccda3d11d794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:29.396890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b50f9-9b4c-491c-9ea6-ccda3d11d794 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:29.397187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b50f9-9b4c-491c-9ea6-ccda3d11d794 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:29.397469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b50f9-9b4c-491c-9ea6-ccda3d11d794 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:29.398130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b50f9-9b4c-491c-9ea6-ccda3d11d794 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:29.398434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b50f9-9b4c-491c-9ea6-ccda3d11d794 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:29.399806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b50f9-9b4c-491c-9ea6-ccda3d11d794 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:29.404369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 439/877] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Aug 24 09:58:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:29.405158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b50f9-9b4c-491c-9ea6-ccda3d11d794 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:29.488236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1f617-8a21-49a8-ba64-da1e1ebf7c9a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:29.506580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:29.510997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b50f9-9b4c-491c-9ea6-ccda3d11d794 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:29.526995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e87232-3772-46a9-b628-72890b64d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:29.527857 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e87232-3772-46a9-b628-72890b64d209 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:29.528175 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e87232-3772-46a9-b628-72890b64d209 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:29.528451 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e87232-3772-46a9-b628-72890b64d209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:29.529206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e87232-3772-46a9-b628-72890b64d209 None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=8b7227998e7846c7a9bc14b12b637ee0) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:29.529514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e87232-3772-46a9-b628-72890b64d209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:29.530941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e87232-3772-46a9-b628-72890b64d209 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:29.548891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 439/878] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:29 2022] GET /identity/v3/role_assignments?user.id=8b7227998e7846c7a9bc14b12b637ee0&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:29.548891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e87232-3772-46a9-b628-72890b64d209 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:29.566717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ff37c-aeb5-41a5-b3bb-397b4a728e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:29.567357 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ff37c-aeb5-41a5-b3bb-397b4a728e3f None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:29.567640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ff37c-aeb5-41a5-b3bb-397b4a728e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:29.567917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ff37c-aeb5-41a5-b3bb-397b4a728e3f None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:29.569371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ff37c-aeb5-41a5-b3bb-397b4a728e3f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:29.571250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ff37c-aeb5-41a5-b3bb-397b4a728e3f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:29.578545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:29.666478 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e87232-3772-46a9-b628-72890b64d209 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:29.683805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 440/879] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:58:29 2022] POST /identity/v3/services => generated 251 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:29.685085 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ff37c-aeb5-41a5-b3bb-397b4a728e3f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:29.727997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86649a2-abb3-4b18-bf0a-1250e10d371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:29.729895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86649a2-abb3-4b18-bf0a-1250e10d371e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:29.730341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86649a2-abb3-4b18-bf0a-1250e10d371e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:29.730622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86649a2-abb3-4b18-bf0a-1250e10d371e None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:29.731547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86649a2-abb3-4b18-bf0a-1250e10d371e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:29.731859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86649a2-abb3-4b18-bf0a-1250e10d371e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:29.734151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86649a2-abb3-4b18-bf0a-1250e10d371e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:29.745004 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 440/880] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Aug 24 09:58:29 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:29.829832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ff37c-aeb5-41a5-b3bb-397b4a728e3f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:29.872072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c94d3-88b2-4b7c-ab38-ef19ca79ffe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:29.872717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c94d3-88b2-4b7c-ab38-ef19ca79ffe8 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:29.872996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c94d3-88b2-4b7c-ab38-ef19ca79ffe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:29.873276 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c94d3-88b2-4b7c-ab38-ef19ca79ffe8 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:29.873950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c94d3-88b2-4b7c-ab38-ef19ca79ffe8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:29.875375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c94d3-88b2-4b7c-ab38-ef19ca79ffe8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:29.977074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 441/881] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:58:29 2022] POST /identity/v3/services => generated 269 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:31.652547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aa21f-c786-4de5-a4eb-eace11fd699c None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:31.653088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aa21f-c786-4de5-a4eb-eace11fd699c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:31.653167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aa21f-c786-4de5-a4eb-eace11fd699c None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:31.654694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 441/882] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:31.661170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72761e52-0c79-49b7-8157-79e82f2f983b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:31.661659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72761e52-0c79-49b7-8157-79e82f2f983b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:31.662125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72761e52-0c79-49b7-8157-79e82f2f983b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:31.684955 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-72761e52-0c79-49b7-8157-79e82f2f983b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:31.686156 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-72761e52-0c79-49b7-8157-79e82f2f983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:31.714885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f9a16-07c2-4c30-b96c-75d2574c746f None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:31.715341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f9a16-07c2-4c30-b96c-75d2574c746f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:31.715739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f9a16-07c2-4c30-b96c-75d2574c746f None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:31.717492 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 442/883] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:58:31.722860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bbcc2-07c7-4ed5-9e5d-ae037dbbe7c5 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:31.723245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bbcc2-07c7-4ed5-9e5d-ae037dbbe7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:31.723625 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bbcc2-07c7-4ed5-9e5d-ae037dbbe7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:31.742466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-fe5bbcc2-07c7-4ed5-9e5d-ae037dbbe7c5 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:31.742700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:31.743909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-fe5bbcc2-07c7-4ed5-9e5d-ae037dbbe7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:31.764940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 442/884] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:31.766183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38587c15-aabe-4e30-a320-b2a015e366ed None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:31.766468 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38587c15-aabe-4e30-a320-b2a015e366ed None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:31.766749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38587c15-aabe-4e30-a320-b2a015e366ed None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:31.768160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 443/885] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:31.774656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f4ae9-cbb3-4662-a04d-54c8923b9c5c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:31.777386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f4ae9-cbb3-4662-a04d-54c8923b9c5c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:31.777869 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f4ae9-cbb3-4662-a04d-54c8923b9c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:31.799426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:31.806124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e75f4ae9-cbb3-4662-a04d-54c8923b9c5c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:31.807424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e75f4ae9-cbb3-4662-a04d-54c8923b9c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:31.844961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 443/886] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:31.880151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 444/887] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:31.951373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb5f21-fa84-485e-aa58-211c54cfc814 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:31.951550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb5f21-fa84-485e-aa58-211c54cfc814 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:31.951826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb5f21-fa84-485e-aa58-211c54cfc814 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:31.953472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 444/888] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:31.958883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb3bfe-36a6-41a4-9855-c9d241d0556d None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:31.959169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb3bfe-36a6-41a4-9855-c9d241d0556d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:31.959509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb3bfe-36a6-41a4-9855-c9d241d0556d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:31.980534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e3cb3bfe-36a6-41a4-9855-c9d241d0556d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:31.981741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e3cb3bfe-36a6-41a4-9855-c9d241d0556d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:32.058795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 445/889] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:32.060015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb5f21-fa84-485e-aa58-211c54cfc814 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:32.117200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb3bfe-36a6-41a4-9855-c9d241d0556d None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:32.174728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb5f21-fa84-485e-aa58-211c54cfc814 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:32.183280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:32.210705 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1549d-b30f-41b2-a4cf-c71ff108c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:32.211264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1549d-b30f-41b2-a4cf-c71ff108c778 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:32.211543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1549d-b30f-41b2-a4cf-c71ff108c778 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:32.211812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1549d-b30f-41b2-a4cf-c71ff108c778 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:32.223939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1549d-b30f-41b2-a4cf-c71ff108c778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-81d1549d-b30f-41b2-a4cf-c71ff108c778 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:32.228931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:32.229843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:32.230722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 445/890] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Aug 24 09:58:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:32.234683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb3bfe-36a6-41a4-9855-c9d241d0556d None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:32.235624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1549d-b30f-41b2-a4cf-c71ff108c778 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:32.271849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:32.274084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbe043-7178-4f30-8d46-f905f9d1128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:32.274692 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbe043-7178-4f30-8d46-f905f9d1128c None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:32.274968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbe043-7178-4f30-8d46-f905f9d1128c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:32.275244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbe043-7178-4f30-8d46-f905f9d1128c None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:32.275892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbe043-7178-4f30-8d46-f905f9d1128c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:32.277289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbe043-7178-4f30-8d46-f905f9d1128c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:32.342971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1549d-b30f-41b2-a4cf-c71ff108c778 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:32.377327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19b24-066d-443f-8e74-752daee6d262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:32.378550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19b24-066d-443f-8e74-752daee6d262 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:32.382502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19b24-066d-443f-8e74-752daee6d262 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:32.382502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19b24-066d-443f-8e74-752daee6d262 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:32.382502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd19b24-066d-443f-8e74-752daee6d262 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:32.382502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd19b24-066d-443f-8e74-752daee6d262 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-ffd19b24-066d-443f-8e74-752daee6d262 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:32.389806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:32.391073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:32.392297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 446/891] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:58:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:32.393304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19b24-066d-443f-8e74-752daee6d262 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:32.482901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:32.490267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19b24-066d-443f-8e74-752daee6d262 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:32.524013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0ef70-3dca-4a87-8a43-c4ed05a2550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:32.524613 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0ef70-3dca-4a87-8a43-c4ed05a2550e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:32.524923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0ef70-3dca-4a87-8a43-c4ed05a2550e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:32.525205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0ef70-3dca-4a87-8a43-c4ed05a2550e None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:32.525861 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0ef70-3dca-4a87-8a43-c4ed05a2550e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:32.526168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0ef70-3dca-4a87-8a43-c4ed05a2550e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:32.527540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0ef70-3dca-4a87-8a43-c4ed05a2550e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:32.535524 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 446/892] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:58:32 2022] POST /identity/v3/services => generated 241 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:58:32.544794 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbe043-7178-4f30-8d46-f905f9d1128c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:32.545024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 447/893] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Aug 24 09:58:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:32.545024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0ef70-3dca-4a87-8a43-c4ed05a2550e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:32.662835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:32.668397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbe043-7178-4f30-8d46-f905f9d1128c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:32.670881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0ef70-3dca-4a87-8a43-c4ed05a2550e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:32.707952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ced71a-385e-46c9-9424-dbc9662382b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:32.708969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ced71a-385e-46c9-9424-dbc9662382b7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:32.709258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ced71a-385e-46c9-9424-dbc9662382b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:32.709520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ced71a-385e-46c9-9424-dbc9662382b7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:32.710129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ced71a-385e-46c9-9424-dbc9662382b7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:32.711807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ced71a-385e-46c9-9424-dbc9662382b7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:32.715010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d76565-0f7b-49fd-aef5-0a3105b03dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:32.715858 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d76565-0f7b-49fd-aef5-0a3105b03dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:32.716320 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d76565-0f7b-49fd-aef5-0a3105b03dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:32.716812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d76565-0f7b-49fd-aef5-0a3105b03dc7 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:32.717673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d76565-0f7b-49fd-aef5-0a3105b03dc7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:32.717982 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d76565-0f7b-49fd-aef5-0a3105b03dc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:32.719343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d76565-0f7b-49fd-aef5-0a3105b03dc7 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-f4ced71a-385e-46c9-9424-dbc9662382b7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:32.720930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:32.722521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:58:32.724216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:32.725767 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 448/895] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:58:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:32.725904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 447/895] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Aug 24 09:58:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:32.726382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d76565-0f7b-49fd-aef5-0a3105b03dc7 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:32.733663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ced71a-385e-46c9-9424-dbc9662382b7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:32.755108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:32.833311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ced71a-385e-46c9-9424-dbc9662382b7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:32.841527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d76565-0f7b-49fd-aef5-0a3105b03dc7 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:32.871531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3dc86-3bac-4950-8318-7729debdca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:32.876776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3dc86-3bac-4950-8318-7729debdca27 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:32.876776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3dc86-3bac-4950-8318-7729debdca27 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:32.876776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3dc86-3bac-4950-8318-7729debdca27 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:32.876776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3dc86-3bac-4950-8318-7729debdca27 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:32.876776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3dc86-3bac-4950-8318-7729debdca27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:32.876776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3dc86-3bac-4950-8318-7729debdca27 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:32.879988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ac417-2375-4fe4-96f5-fecd8e9d7c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:32.880621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ac417-2375-4fe4-96f5-fecd8e9d7c60 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:32.880920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ac417-2375-4fe4-96f5-fecd8e9d7c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:32.881198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ac417-2375-4fe4-96f5-fecd8e9d7c60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:32.882313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 448/896] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Aug 24 09:58:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:32.883475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3dc86-3bac-4950-8318-7729debdca27 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:32.892254 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ac417-2375-4fe4-96f5-fecd8e9d7c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:32.893983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ac417-2375-4fe4-96f5-fecd8e9d7c60 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-660ac417-2375-4fe4-96f5-fecd8e9d7c60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:32.909305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:32.918018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:32.919169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:32.919169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:32.919169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:32.919169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 449/897] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:32.919169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ac417-2375-4fe4-96f5-fecd8e9d7c60 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:32.935305 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:32.993570 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:33.006057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3dc86-3bac-4950-8318-7729debdca27 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:33.028430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ac417-2375-4fe4-96f5-fecd8e9d7c60 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:33.043691 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1ec59-a80b-47dd-974c-8e00b52d4da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:33.044304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1ec59-a80b-47dd-974c-8e00b52d4da1 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:33.044587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1ec59-a80b-47dd-974c-8e00b52d4da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:33.044885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1ec59-a80b-47dd-974c-8e00b52d4da1 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:33.045516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1ec59-a80b-47dd-974c-8e00b52d4da1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:33.045820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1ec59-a80b-47dd-974c-8e00b52d4da1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:33.047184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1ec59-a80b-47dd-974c-8e00b52d4da1 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:33.052682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 449/898] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:58:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:33.053517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1ec59-a80b-47dd-974c-8e00b52d4da1 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:33.071212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe57eb-c60f-436d-86ed-1e399bc4b806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:33.072104 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe57eb-c60f-436d-86ed-1e399bc4b806 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:33.072528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe57eb-c60f-436d-86ed-1e399bc4b806 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:33.072990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe57eb-c60f-436d-86ed-1e399bc4b806 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:33.073670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe57eb-c60f-436d-86ed-1e399bc4b806 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:33.073979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe57eb-c60f-436d-86ed-1e399bc4b806 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:33.075404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe57eb-c60f-436d-86ed-1e399bc4b806 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:33.080441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 450/899] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Aug 24 09:58:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:33.082272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe57eb-c60f-436d-86ed-1e399bc4b806 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:33.166096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:33.175720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1ec59-a80b-47dd-974c-8e00b52d4da1 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:33.191727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe57eb-c60f-436d-86ed-1e399bc4b806 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:33.213445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134bada-5823-4c76-8c2a-73aec04a703b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:33.214086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134bada-5823-4c76-8c2a-73aec04a703b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:33.214362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134bada-5823-4c76-8c2a-73aec04a703b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:33.214629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134bada-5823-4c76-8c2a-73aec04a703b None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:33.215265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134bada-5823-4c76-8c2a-73aec04a703b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:33.215561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134bada-5823-4c76-8c2a-73aec04a703b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:33.216925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134bada-5823-4c76-8c2a-73aec04a703b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:33.229384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76987d0-3316-49c7-941e-63cfa32437ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:33.229864 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 450/900] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:33.230368 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76987d0-3316-49c7-941e-63cfa32437ea None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:33.230685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134bada-5823-4c76-8c2a-73aec04a703b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:33.230981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76987d0-3316-49c7-941e-63cfa32437ea None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:33.231419 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76987d0-3316-49c7-941e-63cfa32437ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:33.232121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76987d0-3316-49c7-941e-63cfa32437ea None admin] RBAC: Adding query filter params (interface=public, service_id=d3a62d3352564045872735d3687fdf26, region_id=RegionOne) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:33.232423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76987d0-3316-49c7-941e-63cfa32437ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:33.233811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76987d0-3316-49c7-941e-63cfa32437ea None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:33.236792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:33.241100 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 451/901] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Aug 24 09:58:33 2022] GET /identity/v3/endpoints?service_id=d3a62d3352564045872735d3687fdf26&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:33.242089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76987d0-3316-49c7-941e-63cfa32437ea None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:33.352804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76987d0-3316-49c7-941e-63cfa32437ea None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:33.360803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134bada-5823-4c76-8c2a-73aec04a703b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:33.391532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff163bb-72a6-4625-a628-16af56100dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:33.392487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff163bb-72a6-4625-a628-16af56100dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:33.393718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff163bb-72a6-4625-a628-16af56100dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:33.393870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff163bb-72a6-4625-a628-16af56100dc2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:33.398370 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc663c-ff43-413d-b941-03048dc630cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:33.399057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc663c-ff43-413d-b941-03048dc630cf None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:33.399346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc663c-ff43-413d-b941-03048dc630cf None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:33.399621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc663c-ff43-413d-b941-03048dc630cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:33.400325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc663c-ff43-413d-b941-03048dc630cf None admin] RBAC: Adding query filter params (interface=public, service_id=5a254af7d66c45be903380bf731ed9b3, region_id=RegionOne) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:33.400637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc663c-ff43-413d-b941-03048dc630cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:33.401959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff163bb-72a6-4625-a628-16af56100dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:33.402064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc663c-ff43-413d-b941-03048dc630cf None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:33.403473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff163bb-72a6-4625-a628-16af56100dc2 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:33.411386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 451/902] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Aug 24 09:58:33 2022] GET /identity/v3/endpoints?service_id=5a254af7d66c45be903380bf731ed9b3&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:33.412265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc663c-ff43-413d-b941-03048dc630cf None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-aff163bb-72a6-4625-a628-16af56100dc2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:33.421266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:33.422256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:33.423216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:33.423216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:33.423216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:33.423216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:33.423216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:33.423216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:33.423216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:33.423216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:33.423216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:33.423216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:33.423216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 452/903] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Aug 24 09:58:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:33.423815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff163bb-72a6-4625-a628-16af56100dc2 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:33.437663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:33.495282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:33.533059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc663c-ff43-413d-b941-03048dc630cf None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:33.540504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff163bb-72a6-4625-a628-16af56100dc2 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:33.570422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a193b79-b991-4567-9821-2d288a0a17a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:33.570945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a193b79-b991-4567-9821-2d288a0a17a8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:33.571206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a193b79-b991-4567-9821-2d288a0a17a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:33.571474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a193b79-b991-4567-9821-2d288a0a17a8 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:33.572074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a193b79-b991-4567-9821-2d288a0a17a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:33.573468 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a193b79-b991-4567-9821-2d288a0a17a8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:33.578422 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c98ad-d770-4c2c-88aa-ed47e386e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:33.579034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c98ad-d770-4c2c-88aa-ed47e386e21d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:33.579314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c98ad-d770-4c2c-88aa-ed47e386e21d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:33.588904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c98ad-d770-4c2c-88aa-ed47e386e21d None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:33.589147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 452/904] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Aug 24 09:58:33 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:58:33.589263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c98ad-d770-4c2c-88aa-ed47e386e21d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:33.589367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a193b79-b991-4567-9821-2d288a0a17a8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:33.589474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c98ad-d770-4c2c-88aa-ed47e386e21d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:33.601557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 453/905] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Aug 24 09:58:33 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:58:33.687369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:33.714986 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a193b79-b991-4567-9821-2d288a0a17a8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:33.759075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4fb5f-f934-4477-bfd2-6998cd9d0bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:33.759701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4fb5f-f934-4477-bfd2-6998cd9d0bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:33.759989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4fb5f-f934-4477-bfd2-6998cd9d0bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:33.760267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4fb5f-f934-4477-bfd2-6998cd9d0bd6 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:33.761048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4fb5f-f934-4477-bfd2-6998cd9d0bd6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:33.761372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4fb5f-f934-4477-bfd2-6998cd9d0bd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:33.762857 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4fb5f-f934-4477-bfd2-6998cd9d0bd6 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:33.784277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 453/906] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Aug 24 09:58:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:33.790246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c98ad-d770-4c2c-88aa-ed47e386e21d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:33.903550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c98ad-d770-4c2c-88aa-ed47e386e21d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:33.928810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:33.941107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc091d-12af-47af-8db2-e5489b20df2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:33.942965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc091d-12af-47af-8db2-e5489b20df2a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:33.942965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc091d-12af-47af-8db2-e5489b20df2a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:33.942965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc091d-12af-47af-8db2-e5489b20df2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:34.014139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc091d-12af-47af-8db2-e5489b20df2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:34.015659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc091d-12af-47af-8db2-e5489b20df2a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:34.123091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 454/907] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:33 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:58:34.130802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4fb5f-f934-4477-bfd2-6998cd9d0bd6 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:34.236263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4fb5f-f934-4477-bfd2-6998cd9d0bd6 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:34.273645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd78f67-48d1-4994-9ffd-fa1120f736fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:34.274272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd78f67-48d1-4994-9ffd-fa1120f736fb None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:34.274548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd78f67-48d1-4994-9ffd-fa1120f736fb None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:34.274824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd78f67-48d1-4994-9ffd-fa1120f736fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:34.286489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd78f67-48d1-4994-9ffd-fa1120f736fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:34.287937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd78f67-48d1-4994-9ffd-fa1120f736fb None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:34.296812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-9fd78f67-48d1-4994-9ffd-fa1120f736fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:34.306881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:34.315473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:34.316782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:34.316782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:34.316782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:34.316782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 454/908] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:34.317851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc091d-12af-47af-8db2-e5489b20df2a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:34.341239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:34.422322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc091d-12af-47af-8db2-e5489b20df2a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:34.459300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725eea7e-c915-4371-881d-39a7befd22d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:34.460143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725eea7e-c915-4371-881d-39a7befd22d4 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:34.460542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725eea7e-c915-4371-881d-39a7befd22d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:34.461055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725eea7e-c915-4371-881d-39a7befd22d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:34.461957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725eea7e-c915-4371-881d-39a7befd22d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:34.462387 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725eea7e-c915-4371-881d-39a7befd22d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:34.464370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725eea7e-c915-4371-881d-39a7befd22d4 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:34.479280 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 455/909] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:34.484527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd78f67-48d1-4994-9ffd-fa1120f736fb None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:34.566944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840cf3e-087e-4b09-a133-f9502a0b6107 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:34.567246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840cf3e-087e-4b09-a133-f9502a0b6107 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:34.567407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840cf3e-087e-4b09-a133-f9502a0b6107 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:34.569502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 456/910] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:34.576010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf907c17-5289-4b06-a61f-2c86c17d718e None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:34.577101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf907c17-5289-4b06-a61f-2c86c17d718e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:34.577390 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf907c17-5289-4b06-a61f-2c86c17d718e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:34.602385 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-cf907c17-5289-4b06-a61f-2c86c17d718e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:34.604597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cf907c17-5289-4b06-a61f-2c86c17d718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:34.606353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd78f67-48d1-4994-9ffd-fa1120f736fb None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:34.647227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a9651-2745-464f-b5e4-e2ff44468845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:34.647853 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a9651-2745-464f-b5e4-e2ff44468845 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:34.648130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a9651-2745-464f-b5e4-e2ff44468845 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:34.648398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a9651-2745-464f-b5e4-e2ff44468845 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:34.655980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a9651-2745-464f-b5e4-e2ff44468845 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:34.657454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a9651-2745-464f-b5e4-e2ff44468845 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-e81a9651-2745-464f-b5e4-e2ff44468845 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:34.670165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:34.671095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:34.672112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:34.672112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:34.672112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:34.672112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:34.672112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:34.672112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:34.672112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:34.672112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:34.672112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 455/911] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:34.677321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a9651-2745-464f-b5e4-e2ff44468845 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:34.705192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 457/912] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:34.728460 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:34.781469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a9651-2745-464f-b5e4-e2ff44468845 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:34.828203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90186807-f3dd-4ba5-ba30-ac39a7da07ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:34.828738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186807-f3dd-4ba5-ba30-ac39a7da07ed None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:34.829047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186807-f3dd-4ba5-ba30-ac39a7da07ed None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:34.829320 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186807-f3dd-4ba5-ba30-ac39a7da07ed None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:34.830024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90186807-f3dd-4ba5-ba30-ac39a7da07ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:34.830489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90186807-f3dd-4ba5-ba30-ac39a7da07ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:34.832454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90186807-f3dd-4ba5-ba30-ac39a7da07ed None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:34.847158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 456/913] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:34.853329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf907c17-5289-4b06-a61f-2c86c17d718e None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:34.869589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:34.961383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf907c17-5289-4b06-a61f-2c86c17d718e None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:34.994678 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90186807-f3dd-4ba5-ba30-ac39a7da07ed None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:35.000269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dae93-bf7b-41a0-aac9-5ae7a8b636d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:35.000902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dae93-bf7b-41a0-aac9-5ae7a8b636d6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:35.001188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dae93-bf7b-41a0-aac9-5ae7a8b636d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:35.001464 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dae93-bf7b-41a0-aac9-5ae7a8b636d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:35.002196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14dae93-bf7b-41a0-aac9-5ae7a8b636d6 None admin] RBAC: Adding query filter params (role.id=47c08bf673724b40aa00a201adb1411b, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=8b7227998e7846c7a9bc14b12b637ee0) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:35.002539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14dae93-bf7b-41a0-aac9-5ae7a8b636d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:35.004193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14dae93-bf7b-41a0-aac9-5ae7a8b636d6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:35.010397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 458/914] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:34 2022] GET /identity/v3/role_assignments?user.id=8b7227998e7846c7a9bc14b12b637ee0&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=47c08bf673724b40aa00a201adb1411b => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:35.119833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90186807-f3dd-4ba5-ba30-ac39a7da07ed None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:35.133402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:35.163526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddeffa-03cd-4ed4-ba9c-2495e4d44986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:35.164238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddeffa-03cd-4ed4-ba9c-2495e4d44986 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:35.164525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddeffa-03cd-4ed4-ba9c-2495e4d44986 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:35.164801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddeffa-03cd-4ed4-ba9c-2495e4d44986 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:35.165428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddeffa-03cd-4ed4-ba9c-2495e4d44986 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:35.166984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddeffa-03cd-4ed4-ba9c-2495e4d44986 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-80ddeffa-03cd-4ed4-ba9c-2495e4d44986 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:35.178669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:35.179707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:35.180534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 457/915] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Aug 24 09:58:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:35.189734 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dae93-bf7b-41a0-aac9-5ae7a8b636d6 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:35.294324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:35.313144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dae93-bf7b-41a0-aac9-5ae7a8b636d6 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:35.353246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93760d9d-52ac-4734-b9b1-07a65e65c5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:35.353865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93760d9d-52ac-4734-b9b1-07a65e65c5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:35.354145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93760d9d-52ac-4734-b9b1-07a65e65c5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:35.354419 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93760d9d-52ac-4734-b9b1-07a65e65c5d3 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:35.355079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93760d9d-52ac-4734-b9b1-07a65e65c5d3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:35.355455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93760d9d-52ac-4734-b9b1-07a65e65c5d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:35.356837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93760d9d-52ac-4734-b9b1-07a65e65c5d3 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:35.366881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 459/916] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Aug 24 09:58:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:35.370018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddeffa-03cd-4ed4-ba9c-2495e4d44986 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:35.480481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddeffa-03cd-4ed4-ba9c-2495e4d44986 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:35.517376 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31aa53-73e4-4058-874b-a6ea5968ddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:35.518585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31aa53-73e4-4058-874b-a6ea5968ddf1 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:35.518870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31aa53-73e4-4058-874b-a6ea5968ddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:35.519140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31aa53-73e4-4058-874b-a6ea5968ddf1 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:35.519772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31aa53-73e4-4058-874b-a6ea5968ddf1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:35.520071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31aa53-73e4-4058-874b-a6ea5968ddf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:35.521466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31aa53-73e4-4058-874b-a6ea5968ddf1 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:35.526839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 458/917] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Aug 24 09:58:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:35.532608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93760d9d-52ac-4734-b9b1-07a65e65c5d3 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:35.645509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93760d9d-52ac-4734-b9b1-07a65e65c5d3 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:35.683417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786aa03-420d-4aa0-a7fc-6cbe375830d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:35.684021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786aa03-420d-4aa0-a7fc-6cbe375830d7 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:35.684297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786aa03-420d-4aa0-a7fc-6cbe375830d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:35.684565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786aa03-420d-4aa0-a7fc-6cbe375830d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:35.685295 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786aa03-420d-4aa0-a7fc-6cbe375830d7 None admin] RBAC: Adding query filter params (interface=public, service_id=319b380edaa24ffab01b81390cb4cd51, region_id=RegionOne) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:35.685598 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786aa03-420d-4aa0-a7fc-6cbe375830d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:35.686958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786aa03-420d-4aa0-a7fc-6cbe375830d7 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:35.695144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c931f-8907-4a03-b722-41b2686ecf61 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:35.695431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c931f-8907-4a03-b722-41b2686ecf61 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:35.695708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c931f-8907-4a03-b722-41b2686ecf61 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:35.697220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 460/919] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Aug 24 09:58:35 2022] GET /identity/v3/endpoints?service_id=319b380edaa24ffab01b81390cb4cd51&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:35.697334 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 459/919] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:35.699337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786aa03-420d-4aa0-a7fc-6cbe375830d7 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:35.702551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d735d-fe87-492e-be62-f9173e320b1b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:35.702832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d735d-fe87-492e-be62-f9173e320b1b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:35.703109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d735d-fe87-492e-be62-f9173e320b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:35.709326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:35.722240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-386d735d-fe87-492e-be62-f9173e320b1b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:35.723559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-386d735d-fe87-492e-be62-f9173e320b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:35.755948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:35.804921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 460/920] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:35.804921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359bd0a-076b-441e-966e-0b4953e1b1bd None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:35.804921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359bd0a-076b-441e-966e-0b4953e1b1bd None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:35.804921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359bd0a-076b-441e-966e-0b4953e1b1bd None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:35.808891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 461/921] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:35.811397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce96226-47ab-4e27-821e-30cd84167c21 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:35.811680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce96226-47ab-4e27-821e-30cd84167c21 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:35.811951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce96226-47ab-4e27-821e-30cd84167c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:35.818757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786aa03-420d-4aa0-a7fc-6cbe375830d7 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:35.841002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-7ce96226-47ab-4e27-821e-30cd84167c21 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:35.843297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7ce96226-47ab-4e27-821e-30cd84167c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:35.859958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37f9ff-1664-4bf9-933e-7958467afb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:35.860604 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37f9ff-1664-4bf9-933e-7958467afb94 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:35.860829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37f9ff-1664-4bf9-933e-7958467afb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:35.861136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37f9ff-1664-4bf9-933e-7958467afb94 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:35.861949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37f9ff-1664-4bf9-933e-7958467afb94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:35.863593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37f9ff-1664-4bf9-933e-7958467afb94 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:35.872897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 461/922] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Aug 24 09:58:35 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:58:35.937087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 462/923] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:36.083313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37f9ff-1664-4bf9-933e-7958467afb94 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:36.161118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:36.194155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37f9ff-1664-4bf9-933e-7958467afb94 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:36.217425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce96226-47ab-4e27-821e-30cd84167c21 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:36.231626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda86beb-954d-4e5e-8f10-a67c7c49e4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:36.232261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda86beb-954d-4e5e-8f10-a67c7c49e4de None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:36.232541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda86beb-954d-4e5e-8f10-a67c7c49e4de None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:36.232814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda86beb-954d-4e5e-8f10-a67c7c49e4de None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:36.233443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda86beb-954d-4e5e-8f10-a67c7c49e4de None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:36.234811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda86beb-954d-4e5e-8f10-a67c7c49e4de None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-fda86beb-954d-4e5e-8f10-a67c7c49e4de None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:36.244531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:36.245790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:36.247219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 462/924] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:58:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:36.251789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda86beb-954d-4e5e-8f10-a67c7c49e4de None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:36.341138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:36.349194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce96226-47ab-4e27-821e-30cd84167c21 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:36.360382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda86beb-954d-4e5e-8f10-a67c7c49e4de None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:36.396489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6baf2-3cd8-4f65-ad78-dc83b0822876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:36.397167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6baf2-3cd8-4f65-ad78-dc83b0822876 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:36.397534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6baf2-3cd8-4f65-ad78-dc83b0822876 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:36.397817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6baf2-3cd8-4f65-ad78-dc83b0822876 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:36.398586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6baf2-3cd8-4f65-ad78-dc83b0822876 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:36.400021 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6baf2-3cd8-4f65-ad78-dc83b0822876 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:36.403526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4ecd6-7e80-4b90-99aa-387ffd97fb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:36.404634 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ecd6-7e80-4b90-99aa-387ffd97fb23 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:36.405166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ecd6-7e80-4b90-99aa-387ffd97fb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:36.405621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ecd6-7e80-4b90-99aa-387ffd97fb23 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:36.406430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4ecd6-7e80-4b90-99aa-387ffd97fb23 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:36.406911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4ecd6-7e80-4b90-99aa-387ffd97fb23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:36.409034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4ecd6-7e80-4b90-99aa-387ffd97fb23 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:36.410726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-f6c6baf2-3cd8-4f65-ad78-dc83b0822876 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:36.410836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:36.411753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:36.412706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 463/925] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Aug 24 09:58:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:36.418170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6baf2-3cd8-4f65-ad78-dc83b0822876 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:36.420030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 463/926] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:58:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:36.428319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4ecd6-7e80-4b90-99aa-387ffd97fb23 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:36.544311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6baf2-3cd8-4f65-ad78-dc83b0822876 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:36.573028 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4ecd6-7e80-4b90-99aa-387ffd97fb23 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:36.582654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f64ed-f255-47fd-8d90-e1187bb33117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:36.583583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f64ed-f255-47fd-8d90-e1187bb33117 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:36.583707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f64ed-f255-47fd-8d90-e1187bb33117 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:36.584017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f64ed-f255-47fd-8d90-e1187bb33117 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:36.584805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f64ed-f255-47fd-8d90-e1187bb33117 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:36.585215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f64ed-f255-47fd-8d90-e1187bb33117 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:36.586663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f64ed-f255-47fd-8d90-e1187bb33117 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:36.592269 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 464/927] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Aug 24 09:58:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:36.600909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f64ed-f255-47fd-8d90-e1187bb33117 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:36.621335 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c126b-f781-4628-977b-18a3b8dee7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:36.622144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c126b-f781-4628-977b-18a3b8dee7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:36.622788 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c126b-f781-4628-977b-18a3b8dee7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:36.623167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c126b-f781-4628-977b-18a3b8dee7c5 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:36.623927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c126b-f781-4628-977b-18a3b8dee7c5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:36.624486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c126b-f781-4628-977b-18a3b8dee7c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:36.626348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c126b-f781-4628-977b-18a3b8dee7c5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:36.633562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 464/928] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Aug 24 09:58:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:36.637724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:36.639396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c126b-f781-4628-977b-18a3b8dee7c5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:36.711037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:36.746178 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f64ed-f255-47fd-8d90-e1187bb33117 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:36.763991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c126b-f781-4628-977b-18a3b8dee7c5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:36.794127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8e533-eafd-4120-a151-3731c1964dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:36.795564 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8e533-eafd-4120-a151-3731c1964dcd None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:36.796251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8e533-eafd-4120-a151-3731c1964dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:36.796676 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8e533-eafd-4120-a151-3731c1964dcd None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:36.798075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8e533-eafd-4120-a151-3731c1964dcd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:36.798652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8e533-eafd-4120-a151-3731c1964dcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:36.801189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8e533-eafd-4120-a151-3731c1964dcd None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:36.802901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c850a3a-e8e1-4ff2-a737-d65ce89091cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:36.803776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c850a3a-e8e1-4ff2-a737-d65ce89091cf None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:36.804156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c850a3a-e8e1-4ff2-a737-d65ce89091cf None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:36.804758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c850a3a-e8e1-4ff2-a737-d65ce89091cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:36.805417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c850a3a-e8e1-4ff2-a737-d65ce89091cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:36.806878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c850a3a-e8e1-4ff2-a737-d65ce89091cf None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:36.813192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 465/929] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Aug 24 09:58:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:36.816338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8e533-eafd-4120-a151-3731c1964dcd None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:36.912951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 465/930] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 09:58:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:36.918461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:36.933296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8e533-eafd-4120-a151-3731c1964dcd None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:36.979439 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a436-4647-4232-a897-9bac4165477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:36.980461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a436-4647-4232-a897-9bac4165477c None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:36.981021 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a436-4647-4232-a897-9bac4165477c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:36.981573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a436-4647-4232-a897-9bac4165477c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:36.982513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629a436-4647-4232-a897-9bac4165477c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:36.984158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629a436-4647-4232-a897-9bac4165477c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:37.005007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032ab32-cbc9-4c21-9c2b-2e5d4c4cf41a None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:37.005351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032ab32-cbc9-4c21-9c2b-2e5d4c4cf41a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:37.005631 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032ab32-cbc9-4c21-9c2b-2e5d4c4cf41a None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:37.007229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 466/931] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:58:37.012757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ff7c6-8792-4bce-971f-9071c820cb83 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:37.013055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ff7c6-8792-4bce-971f-9071c820cb83 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:37.013334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ff7c6-8792-4bce-971f-9071c820cb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:37.032339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b78ff7c6-8792-4bce-971f-9071c820cb83 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:37.034128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b78ff7c6-8792-4bce-971f-9071c820cb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:37.114267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 466/932] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 09:58:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:58:37.121896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 467/933] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:37.401576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a436-4647-4232-a897-9bac4165477c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:37.508096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a436-4647-4232-a897-9bac4165477c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:37.547683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7914889-caad-43df-b34c-d62ecf892f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:37.548321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7914889-caad-43df-b34c-d62ecf892f3e None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:37.548603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7914889-caad-43df-b34c-d62ecf892f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:37.548897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7914889-caad-43df-b34c-d62ecf892f3e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:37.559169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:37.563513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7914889-caad-43df-b34c-d62ecf892f3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-c7914889-caad-43df-b34c-d62ecf892f3e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:37.568677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:37.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:37.574442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 467/934] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Aug 24 09:58:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:37.576143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ff7c6-8792-4bce-971f-9071c820cb83 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:37.626554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:37.687876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ff7c6-8792-4bce-971f-9071c820cb83 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:37.725003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525f117-51db-47fa-877b-50a02a4e6963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:37.725905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525f117-51db-47fa-877b-50a02a4e6963 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:37.726358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525f117-51db-47fa-877b-50a02a4e6963 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:37.726774 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525f117-51db-47fa-877b-50a02a4e6963 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:37.727573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525f117-51db-47fa-877b-50a02a4e6963 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:37.728023 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525f117-51db-47fa-877b-50a02a4e6963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:37.729537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525f117-51db-47fa-877b-50a02a4e6963 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:37.738471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 468/935] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Aug 24 09:58:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:37.743859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7914889-caad-43df-b34c-d62ecf892f3e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:37.851026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7914889-caad-43df-b34c-d62ecf892f3e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:37.886215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565431d3-5458-480a-9e95-d39baa61fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:37.886846 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565431d3-5458-480a-9e95-d39baa61fdb9 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:37.887130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565431d3-5458-480a-9e95-d39baa61fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:37.887406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565431d3-5458-480a-9e95-d39baa61fdb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:37.899213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565431d3-5458-480a-9e95-d39baa61fdb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:37.900670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565431d3-5458-480a-9e95-d39baa61fdb9 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-565431d3-5458-480a-9e95-d39baa61fdb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:37.916775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:37.918221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:37.919140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:37.919140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:37.919140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:37.919140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 468/936] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:37.924301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525f117-51db-47fa-877b-50a02a4e6963 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:37.957806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26263798-43bc-47dd-896e-8a8253d96a60 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:37.958004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26263798-43bc-47dd-896e-8a8253d96a60 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:37.958284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26263798-43bc-47dd-896e-8a8253d96a60 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:37.959613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 469/937] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:37.965820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700c411-52d4-4615-af0c-22e50d01e9b6 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:37.966121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700c411-52d4-4615-af0c-22e50d01e9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:37.966408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700c411-52d4-4615-af0c-22e50d01e9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:37.990926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-5700c411-52d4-4615-af0c-22e50d01e9b6 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:37.992817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5700c411-52d4-4615-af0c-22e50d01e9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:38.038403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525f117-51db-47fa-877b-50a02a4e6963 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:38.046733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:38.058545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:38.082101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc26345-d445-407e-8a6b-38087fa2dbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:38.083048 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc26345-d445-407e-8a6b-38087fa2dbce None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:38.083515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc26345-d445-407e-8a6b-38087fa2dbce None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:38.084097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc26345-d445-407e-8a6b-38087fa2dbce None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:38.084205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 470/938] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:38.085092 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc26345-d445-407e-8a6b-38087fa2dbce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:38.085607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc26345-d445-407e-8a6b-38087fa2dbce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:38.087281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc26345-d445-407e-8a6b-38087fa2dbce None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:38.105304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 469/939] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:38.116911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700c411-52d4-4615-af0c-22e50d01e9b6 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:38.224929 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700c411-52d4-4615-af0c-22e50d01e9b6 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:38.259380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51b075-3a0c-4948-9af5-dba00f9a7763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:38.260034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b075-3a0c-4948-9af5-dba00f9a7763 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:38.260315 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b075-3a0c-4948-9af5-dba00f9a7763 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:38.260585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b075-3a0c-4948-9af5-dba00f9a7763 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:38.272905 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51b075-3a0c-4948-9af5-dba00f9a7763 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:38.272905 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51b075-3a0c-4948-9af5-dba00f9a7763 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-5b51b075-3a0c-4948-9af5-dba00f9a7763 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:38.284911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:38.286134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:38.287126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:38.287126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:38.287126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:38.287126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:38.287126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:38.287126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:38.287126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:38.287126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:38.287126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:38.287126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:38.287126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 471/940] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Aug 24 09:58:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:38.292089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc26345-d445-407e-8a6b-38087fa2dbce None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:38.371127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51b075-3a0c-4948-9af5-dba00f9a7763 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:38.405940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc26345-d445-407e-8a6b-38087fa2dbce None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:38.417665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:38.444049 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999364fe-7468-460e-84b0-ae6d686b3780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:38.444666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999364fe-7468-460e-84b0-ae6d686b3780 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:38.445015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999364fe-7468-460e-84b0-ae6d686b3780 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:38.445306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999364fe-7468-460e-84b0-ae6d686b3780 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:38.446130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999364fe-7468-460e-84b0-ae6d686b3780 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:38.446450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999364fe-7468-460e-84b0-ae6d686b3780 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:38.447887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999364fe-7468-460e-84b0-ae6d686b3780 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:38.474365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 470/941] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Aug 24 09:58:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:38.475589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999364fe-7468-460e-84b0-ae6d686b3780 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:38.479223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51b075-3a0c-4948-9af5-dba00f9a7763 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:38.505310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:38.515516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5b207-75ff-41f7-80ae-5047178bd6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:38.516148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5b207-75ff-41f7-80ae-5047178bd6fd None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:38.516428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5b207-75ff-41f7-80ae-5047178bd6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:38.516698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5b207-75ff-41f7-80ae-5047178bd6fd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:38.517312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5b207-75ff-41f7-80ae-5047178bd6fd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:38.519348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5b207-75ff-41f7-80ae-5047178bd6fd None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-41c5b207-75ff-41f7-80ae-5047178bd6fd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:38.528577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:38.529640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:38.530498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 472/942] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Aug 24 09:58:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:38.536326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5b207-75ff-41f7-80ae-5047178bd6fd None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:38.591772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999364fe-7468-460e-84b0-ae6d686b3780 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:38.640832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aad0c-919e-480a-9c0e-e0e235acaeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:38.641644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aad0c-919e-480a-9c0e-e0e235acaeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:38.642016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aad0c-919e-480a-9c0e-e0e235acaeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:38.642435 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aad0c-919e-480a-9c0e-e0e235acaeb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:38.656449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5b207-75ff-41f7-80ae-5047178bd6fd None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:38.657610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087aad0c-919e-480a-9c0e-e0e235acaeb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:38.660574 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087aad0c-919e-480a-9c0e-e0e235acaeb5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:38.664619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-087aad0c-919e-480a-9c0e-e0e235acaeb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:38.677520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:38.680397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:38.682540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:38.682540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:38.682540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:38.688896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 471/943] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:38.688896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aad0c-919e-480a-9c0e-e0e235acaeb5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:38.695960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d72bff-c4a9-4fce-9edb-224eb9a4d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:38.696665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d72bff-c4a9-4fce-9edb-224eb9a4d054 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:38.696990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d72bff-c4a9-4fce-9edb-224eb9a4d054 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:38.697270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d72bff-c4a9-4fce-9edb-224eb9a4d054 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:38.697903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d72bff-c4a9-4fce-9edb-224eb9a4d054 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:38.698214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d72bff-c4a9-4fce-9edb-224eb9a4d054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:38.699906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d72bff-c4a9-4fce-9edb-224eb9a4d054 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:38.706471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 473/944] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Aug 24 09:58:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:38.711952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d72bff-c4a9-4fce-9edb-224eb9a4d054 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:38.777120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:38.802010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aad0c-919e-480a-9c0e-e0e235acaeb5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:38.825071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d72bff-c4a9-4fce-9edb-224eb9a4d054 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:38.839851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d434c2-6395-4498-bea0-0ad82fd15555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:38.840671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d434c2-6395-4498-bea0-0ad82fd15555 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:38.841096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d434c2-6395-4498-bea0-0ad82fd15555 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:38.841528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d434c2-6395-4498-bea0-0ad82fd15555 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:38.842284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d434c2-6395-4498-bea0-0ad82fd15555 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:38.842707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d434c2-6395-4498-bea0-0ad82fd15555 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:38.844237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d434c2-6395-4498-bea0-0ad82fd15555 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:38.857450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 472/945] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:38.863651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d434c2-6395-4498-bea0-0ad82fd15555 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:38.866483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9502bc7-16a7-444a-b1ad-c5fea95954e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:38.867296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9502bc7-16a7-444a-b1ad-c5fea95954e0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:38.867672 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9502bc7-16a7-444a-b1ad-c5fea95954e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:38.868057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9502bc7-16a7-444a-b1ad-c5fea95954e0 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:38.868753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9502bc7-16a7-444a-b1ad-c5fea95954e0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:38.869207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9502bc7-16a7-444a-b1ad-c5fea95954e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:38.870697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9502bc7-16a7-444a-b1ad-c5fea95954e0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:38.876553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 474/946] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Aug 24 09:58:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:38.885982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9502bc7-16a7-444a-b1ad-c5fea95954e0 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:38.942286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:38.985193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d434c2-6395-4498-bea0-0ad82fd15555 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:39.012974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9502bc7-16a7-444a-b1ad-c5fea95954e0 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:39.023809 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada58b3-e32f-4f47-85a7-91bd0a082112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:39.024628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada58b3-e32f-4f47-85a7-91bd0a082112 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:39.025339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada58b3-e32f-4f47-85a7-91bd0a082112 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:39.025486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada58b3-e32f-4f47-85a7-91bd0a082112 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:39.028639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:39.034421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada58b3-e32f-4f47-85a7-91bd0a082112 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:39.036259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada58b3-e32f-4f47-85a7-91bd0a082112 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-3ada58b3-e32f-4f47-85a7-91bd0a082112 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:39.048950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:39.050036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:39.050939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:39.050939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:39.050939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:39.050939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:39.050939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:39.050939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:39.050939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:39.050939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:39.052944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 473/947] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:39.056897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada58b3-e32f-4f47-85a7-91bd0a082112 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:39.057536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5da3cc-2acf-4ca8-ada1-8e3bf1219f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:39.058377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5da3cc-2acf-4ca8-ada1-8e3bf1219f2b None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:39.058859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5da3cc-2acf-4ca8-ada1-8e3bf1219f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:39.059356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5da3cc-2acf-4ca8-ada1-8e3bf1219f2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:39.060241 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5da3cc-2acf-4ca8-ada1-8e3bf1219f2b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:39.061869 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5da3cc-2acf-4ca8-ada1-8e3bf1219f2b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:39.153420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 475/948] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 09:58:38 2022] POST /identity/v3/endpoints => generated 349 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:39.153420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7be8b-623c-4279-a344-80c898203f91 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:39.153420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7be8b-623c-4279-a344-80c898203f91 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:39.153420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7be8b-623c-4279-a344-80c898203f91 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:39.153420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 476/949] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:39.156362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc3059-ea78-4f24-ae46-f0e0957fbaa0 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:39.156655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc3059-ea78-4f24-ae46-f0e0957fbaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:39.156968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc3059-ea78-4f24-ae46-f0e0957fbaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:39.173333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada58b3-e32f-4f47-85a7-91bd0a082112 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:39.184055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-87bc3059-ea78-4f24-ae46-f0e0957fbaa0 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:39.185328 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-87bc3059-ea78-4f24-ae46-f0e0957fbaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:39.226058 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db351ec9-d406-4649-b188-22a3679f56cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:39.226876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db351ec9-d406-4649-b188-22a3679f56cb None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:39.227090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db351ec9-d406-4649-b188-22a3679f56cb None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:39.227368 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db351ec9-d406-4649-b188-22a3679f56cb None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:39.228095 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db351ec9-d406-4649-b188-22a3679f56cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:39.228401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db351ec9-d406-4649-b188-22a3679f56cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:39.229928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db351ec9-d406-4649-b188-22a3679f56cb None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:39.244763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 474/950] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:39.246087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2eeef-b6eb-49ce-ad7b-89d181f6f5e7 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:39.246398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2eeef-b6eb-49ce-ad7b-89d181f6f5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:39.246647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2eeef-b6eb-49ce-ad7b-89d181f6f5e7 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:39.248138 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 475/951] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:39.252635 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2eeef-b6eb-49ce-ad7b-89d181f6f5e7 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:39.276083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 477/952] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:39.277443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82e6a4-aef7-4449-9023-2efea3ccdeea None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:39.277821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82e6a4-aef7-4449-9023-2efea3ccdeea None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:39.278166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82e6a4-aef7-4449-9023-2efea3ccdeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:39.298645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-8d82e6a4-aef7-4449-9023-2efea3ccdeea None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:39.299873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8d82e6a4-aef7-4449-9023-2efea3ccdeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:39.318120 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:39.332322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:39.386310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 478/953] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:39.399404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2eeef-b6eb-49ce-ad7b-89d181f6f5e7 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:39.435382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be709b-1093-445d-8d39-c54d934dc5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:39.436050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be709b-1093-445d-8d39-c54d934dc5d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:39.436340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be709b-1093-445d-8d39-c54d934dc5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:39.436610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be709b-1093-445d-8d39-c54d934dc5d3 None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/8b7227998e7846c7a9bc14b12b637ee0/roles/47c08bf673724b40aa00a201adb1411b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:39.438879 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be709b-1093-445d-8d39-c54d934dc5d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=8b7227998e7846c7a9bc14b12b637ee0, role_id=47c08bf673724b40aa00a201adb1411b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:39.440974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be709b-1093-445d-8d39-c54d934dc5d3 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:39.541128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 476/954] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:58:39 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/8b7227998e7846c7a9bc14b12b637ee0/roles/47c08bf673724b40aa00a201adb1411b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:58:39.554549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82e6a4-aef7-4449-9023-2efea3ccdeea None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:39.675623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82e6a4-aef7-4449-9023-2efea3ccdeea None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:39.680285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be709b-1093-445d-8d39-c54d934dc5d3 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:39.720084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcf707-6279-4261-bdbe-6d89bb22cd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:39.720725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcf707-6279-4261-bdbe-6d89bb22cd3b None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:39.721145 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcf707-6279-4261-bdbe-6d89bb22cd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:39.721410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcf707-6279-4261-bdbe-6d89bb22cd3b None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:39.722184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcf707-6279-4261-bdbe-6d89bb22cd3b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:39.723620 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcf707-6279-4261-bdbe-6d89bb22cd3b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:39.726281 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: WARNING keystone.server.flask.common [None req-eefcf707-6279-4261-bdbe-6d89bb22cd3b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 09:58:39.773778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.sql.core [None req-eefcf707-6279-4261-bdbe-6d89bb22cd3b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:58:39.773778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 09:58:39.773778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [parameters: {'id': 'c08776ed7190471b971ea2e382c7b69b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 09:58:39.773778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81931) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-eefcf707-6279-4261-bdbe-6d89bb22cd3b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c08776ed7190471b971ea2e382c7b69b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c08776ed7190471b971ea2e382c7b69b. Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:58:39.784974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application result.read() Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 09:58:39.786093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application yield resource Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise self.value Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application session.commit() Aug 24 09:58:39.787049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:58:39.787972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:58:39.788925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application result.read() Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:58:39.789845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [parameters: {'id': 'c08776ed7190471b971ea2e382c7b69b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:39.790735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:39.791565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:39.792531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:39.792531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 09:58:39.792531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 09:58:39.792531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:39.792531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:39.792531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 09:58:39.792531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 09:58:39.792531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c08776ed7190471b971ea2e382c7b69b. Aug 24 09:58:39.792531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:39.792531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 479/955] 10.222.0.30 () {60 vars in 1140 bytes} [Wed Aug 24 09:58:39 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:39.800989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcf707-6279-4261-bdbe-6d89bb22cd3b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:39.805462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be709b-1093-445d-8d39-c54d934dc5d3 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:39.815319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:39.822469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:39.844638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaf3bb-d776-4d64-a054-66b2bd715d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:39.845306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaf3bb-d776-4d64-a054-66b2bd715d5e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:39.845587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaf3bb-d776-4d64-a054-66b2bd715d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:39.845909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaf3bb-d776-4d64-a054-66b2bd715d5e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:39.846567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbaf3bb-d776-4d64-a054-66b2bd715d5e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:39.848101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbaf3bb-d776-4d64-a054-66b2bd715d5e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-cfbaf3bb-d776-4d64-a054-66b2bd715d5e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:39.857338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:39.859689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:39.860740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 477/956] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:39.865418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaf3bb-d776-4d64-a054-66b2bd715d5e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:39.928709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcf707-6279-4261-bdbe-6d89bb22cd3b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:39.967910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a16d-0153-41d9-b6d8-85eb725198dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:39.968556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a16d-0153-41d9-b6d8-85eb725198dd None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:39.968840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a16d-0153-41d9-b6d8-85eb725198dd None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:39.969899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a16d-0153-41d9-b6d8-85eb725198dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:39.978779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20a16d-0153-41d9-b6d8-85eb725198dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:39.980447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20a16d-0153-41d9-b6d8-85eb725198dd None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:39.983191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaf3bb-d776-4d64-a054-66b2bd715d5e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-6f20a16d-0153-41d9-b6d8-85eb725198dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:39.993277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:39.994194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:39.995215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:39.995215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:39.995215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:39.995215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:39.995215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:39.995215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:39.995215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:39.995215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:39.999602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 480/957] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:40.000581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a16d-0153-41d9-b6d8-85eb725198dd None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:40.029165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe322f-2760-4d65-b541-2c472c9e82d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:40.030770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe322f-2760-4d65-b541-2c472c9e82d5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:40.031234 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe322f-2760-4d65-b541-2c472c9e82d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:40.031910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe322f-2760-4d65-b541-2c472c9e82d5 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:40.032687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe322f-2760-4d65-b541-2c472c9e82d5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:40.033874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe322f-2760-4d65-b541-2c472c9e82d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:40.036319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe322f-2760-4d65-b541-2c472c9e82d5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:40.042840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 478/958] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:40.048495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe322f-2760-4d65-b541-2c472c9e82d5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:40.084534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:40.106034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:40.124513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a16d-0153-41d9-b6d8-85eb725198dd None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:40.165667 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583dd57-fc89-4647-a637-637f0f9291ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:40.166668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583dd57-fc89-4647-a637-637f0f9291ba None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:40.167160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583dd57-fc89-4647-a637-637f0f9291ba None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:40.167641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583dd57-fc89-4647-a637-637f0f9291ba None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:40.168613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583dd57-fc89-4647-a637-637f0f9291ba None admin] RBAC: Adding query filter params (name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:40.169230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583dd57-fc89-4647-a637-637f0f9291ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:40.170989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583dd57-fc89-4647-a637-637f0f9291ba None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:40.179231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe322f-2760-4d65-b541-2c472c9e82d5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:40.204549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 481/959] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:40.227875 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd8e6d-570f-4b82-bc06-c366793a627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:40.229010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd8e6d-570f-4b82-bc06-c366793a627d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:40.229649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd8e6d-570f-4b82-bc06-c366793a627d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:40.230146 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd8e6d-570f-4b82-bc06-c366793a627d None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:40.231064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd8e6d-570f-4b82-bc06-c366793a627d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:40.231706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd8e6d-570f-4b82-bc06-c366793a627d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:40.233561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd8e6d-570f-4b82-bc06-c366793a627d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:40.240833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 479/960] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:40 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:41.227384 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d0cb0-740c-4484-b3e3-b26856bbd411 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:41.227907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d0cb0-740c-4484-b3e3-b26856bbd411 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:41.227907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d0cb0-740c-4484-b3e3-b26856bbd411 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:41.229557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 482/961] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:41.235381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585d8fa-3b85-4b9d-9886-583e7a8f1b9a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:41.235827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585d8fa-3b85-4b9d-9886-583e7a8f1b9a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:41.236012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585d8fa-3b85-4b9d-9886-583e7a8f1b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:41.254547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0585d8fa-3b85-4b9d-9886-583e7a8f1b9a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:41.255786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0585d8fa-3b85-4b9d-9886-583e7a8f1b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:41.332819 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 480/962] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:41.603205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d0cb0-740c-4484-b3e3-b26856bbd411 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:41.643399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845aebc-0d33-421a-a547-fb171bf04bc2 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:41.643872 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845aebc-0d33-421a-a547-fb171bf04bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:41.644326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845aebc-0d33-421a-a547-fb171bf04bc2 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:41.645962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 481/963] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:41.651190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc66907-9a8b-4db0-b1bf-e99ab7b28d86 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:41.651468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc66907-9a8b-4db0-b1bf-e99ab7b28d86 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:41.651743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc66907-9a8b-4db0-b1bf-e99ab7b28d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:41.670375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-fdc66907-9a8b-4db0-b1bf-e99ab7b28d86 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:41.671567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-fdc66907-9a8b-4db0-b1bf-e99ab7b28d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:41.701302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:41.725575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d0cb0-740c-4484-b3e3-b26856bbd411 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:41.752984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 482/964] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:41.761243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:41.769560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b61dfc-53a8-4360-936f-7f30ce750bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:41.770250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b61dfc-53a8-4360-936f-7f30ce750bac None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:41.770648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b61dfc-53a8-4360-936f-7f30ce750bac None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:41.771022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b61dfc-53a8-4360-936f-7f30ce750bac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:41.771709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b61dfc-53a8-4360-936f-7f30ce750bac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:41.773216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b61dfc-53a8-4360-936f-7f30ce750bac None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-f5b61dfc-53a8-4360-936f-7f30ce750bac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:41.782865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:41.789591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:41.790437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 483/965] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Aug 24 09:58:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:41.794242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc66907-9a8b-4db0-b1bf-e99ab7b28d86 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:41.902867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc66907-9a8b-4db0-b1bf-e99ab7b28d86 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:41.940168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5d78e-85d6-42af-9d9c-9f0cebbd009a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:41.940785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5d78e-85d6-42af-9d9c-9f0cebbd009a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:41.941099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5d78e-85d6-42af-9d9c-9f0cebbd009a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:41.941368 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5d78e-85d6-42af-9d9c-9f0cebbd009a None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:41.942046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5d78e-85d6-42af-9d9c-9f0cebbd009a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:41.942345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5d78e-85d6-42af-9d9c-9f0cebbd009a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:41.943714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5d78e-85d6-42af-9d9c-9f0cebbd009a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:41.952978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 483/966] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Aug 24 09:58:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:41.955189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b61dfc-53a8-4360-936f-7f30ce750bac None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:42.025630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5d78e-85d6-42af-9d9c-9f0cebbd009a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:42.059464 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:42.068353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b61dfc-53a8-4360-936f-7f30ce750bac None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:42.105235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5a9d8-00d5-40cb-9637-b13a858a8c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:42.105873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5a9d8-00d5-40cb-9637-b13a858a8c75 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.106152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5a9d8-00d5-40cb-9637-b13a858a8c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.106473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5a9d8-00d5-40cb-9637-b13a858a8c75 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.107169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5a9d8-00d5-40cb-9637-b13a858a8c75 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:42.107475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5a9d8-00d5-40cb-9637-b13a858a8c75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:42.108847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5a9d8-00d5-40cb-9637-b13a858a8c75 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:42.121039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 484/967] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Aug 24 09:58:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:42.122631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5a9d8-00d5-40cb-9637-b13a858a8c75 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:42.150784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5d78e-85d6-42af-9d9c-9f0cebbd009a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:42.182539 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:42.188889 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbbc74-a98b-4b51-b95b-43d8c57b3db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:42.189737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbbc74-a98b-4b51-b95b-43d8c57b3db8 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.190244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbbc74-a98b-4b51-b95b-43d8c57b3db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.190606 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbbc74-a98b-4b51-b95b-43d8c57b3db8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.201962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbbc74-a98b-4b51-b95b-43d8c57b3db8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-fbbbbc74-a98b-4b51-b95b-43d8c57b3db8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:42.207474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:42.209676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:58:42.211715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:42.213601 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 484/968] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Aug 24 09:58:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:42.221060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbbc74-a98b-4b51-b95b-43d8c57b3db8 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:42.232402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5a9d8-00d5-40cb-9637-b13a858a8c75 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:42.275308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d9b47-ac50-40fd-8df8-4ceeeb1c3041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:42.275923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d9b47-ac50-40fd-8df8-4ceeeb1c3041 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.276261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d9b47-ac50-40fd-8df8-4ceeeb1c3041 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.276535 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d9b47-ac50-40fd-8df8-4ceeeb1c3041 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.277268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d9b47-ac50-40fd-8df8-4ceeeb1c3041 None admin] RBAC: Adding query filter params (interface=admin, service_id=319b380edaa24ffab01b81390cb4cd51, region_id=RegionOne) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:42.277572 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d9b47-ac50-40fd-8df8-4ceeeb1c3041 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:42.278940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d9b47-ac50-40fd-8df8-4ceeeb1c3041 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:42.285604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 485/969] 10.222.0.30 () {56 vars in 1249 bytes} [Wed Aug 24 09:58:42 2022] GET /identity/v3/endpoints?service_id=319b380edaa24ffab01b81390cb4cd51&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:42.286823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b744cb-1847-465d-ae7a-fd65575d00f5 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.287114 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b744cb-1847-465d-ae7a-fd65575d00f5 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.287391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b744cb-1847-465d-ae7a-fd65575d00f5 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.288980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 486/970] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:42.297451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7878b-fa7c-4b3a-ab5b-dabb15c95c21 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.297737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7878b-fa7c-4b3a-ab5b-dabb15c95c21 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.298012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7878b-fa7c-4b3a-ab5b-dabb15c95c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.313935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:42.319625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-d4b7878b-fa7c-4b3a-ab5b-dabb15c95c21 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:42.321167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d4b7878b-fa7c-4b3a-ab5b-dabb15c95c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:42.326482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbbc74-a98b-4b51-b95b-43d8c57b3db8 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:42.362475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15804c-5af0-4771-9da6-cfd5cbfb10ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:42.363261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15804c-5af0-4771-9da6-cfd5cbfb10ef None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.363542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15804c-5af0-4771-9da6-cfd5cbfb10ef None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.363815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15804c-5af0-4771-9da6-cfd5cbfb10ef None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.364492 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15804c-5af0-4771-9da6-cfd5cbfb10ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:42.364793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15804c-5af0-4771-9da6-cfd5cbfb10ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:42.366195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15804c-5af0-4771-9da6-cfd5cbfb10ef None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:42.375324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 485/971] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Aug 24 09:58:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:42.376216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15804c-5af0-4771-9da6-cfd5cbfb10ef None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:42.396912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 487/972] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:42.396912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a17d7-cdcc-4bdc-9605-89beb61adbe6 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.396912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a17d7-cdcc-4bdc-9605-89beb61adbe6 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.396912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a17d7-cdcc-4bdc-9605-89beb61adbe6 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.396912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 488/973] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:42.396912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a17d7-cdcc-4bdc-9605-89beb61adbe6 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:42.460648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:42.494483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15804c-5af0-4771-9da6-cfd5cbfb10ef None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:42.520822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a17d7-cdcc-4bdc-9605-89beb61adbe6 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:42.531880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1a90e-a8e1-486e-a40b-def7e6ac0a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:42.532582 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1a90e-a8e1-486e-a40b-def7e6ac0a44 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.532839 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1a90e-a8e1-486e-a40b-def7e6ac0a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.533887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1a90e-a8e1-486e-a40b-def7e6ac0a44 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.534722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1a90e-a8e1-486e-a40b-def7e6ac0a44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:42.536332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1a90e-a8e1-486e-a40b-def7e6ac0a44 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:42.543383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 486/974] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Aug 24 09:58:42 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:58:42.544622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6cd53-c155-4999-8004-243fca153fcc None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.548903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6cd53-c155-4999-8004-243fca153fcc None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.548903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6cd53-c155-4999-8004-243fca153fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.568290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-61e6cd53-c155-4999-8004-243fca153fcc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:42.569930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-61e6cd53-c155-4999-8004-243fca153fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:42.576362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f02d3-970e-4793-8c52-98b740c362ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:42.577215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f02d3-970e-4793-8c52-98b740c362ef None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.577615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f02d3-970e-4793-8c52-98b740c362ef None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.578045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f02d3-970e-4793-8c52-98b740c362ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.585085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:42.592539 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f02d3-970e-4793-8c52-98b740c362ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:42.594262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f02d3-970e-4793-8c52-98b740c362ef None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-fd2f02d3-970e-4793-8c52-98b740c362ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:42.613412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:42.614490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:42.615422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:42.615422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:42.615422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:42.615422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 489/975] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:42.624918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f02d3-970e-4793-8c52-98b740c362ef None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:42.685322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 487/976] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:42.698887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6cd53-c155-4999-8004-243fca153fcc None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:42.757830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:42.762820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f02d3-970e-4793-8c52-98b740c362ef None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:42.812118 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece2caf-77d9-4449-8e88-c131e874a3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:42.812740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece2caf-77d9-4449-8e88-c131e874a3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.813058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece2caf-77d9-4449-8e88-c131e874a3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.813339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece2caf-77d9-4449-8e88-c131e874a3b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.813985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece2caf-77d9-4449-8e88-c131e874a3b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:42.814363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece2caf-77d9-4449-8e88-c131e874a3b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:42.816680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece2caf-77d9-4449-8e88-c131e874a3b6 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:42.828730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6cd53-c155-4999-8004-243fca153fcc None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:42.835399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 490/977] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:42.840552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece2caf-77d9-4449-8e88-c131e874a3b6 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:42.890411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af8bec-499c-4885-a73e-d0e8b8651a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:42.891253 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af8bec-499c-4885-a73e-d0e8b8651a9d None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:42.891677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af8bec-499c-4885-a73e-d0e8b8651a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:42.892051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af8bec-499c-4885-a73e-d0e8b8651a9d None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:42.897578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af8bec-499c-4885-a73e-d0e8b8651a9d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:42.897578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af8bec-499c-4885-a73e-d0e8b8651a9d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:42.971072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece2caf-77d9-4449-8e88-c131e874a3b6 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:42.980168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 488/978] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:58:42 2022] POST /identity/v3/services => generated 242 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:58:42.980983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af8bec-499c-4885-a73e-d0e8b8651a9d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:42.984883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:43.025332 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ec449-0e57-49c7-af80-aa886f7eae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:43.025907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ec449-0e57-49c7-af80-aa886f7eae83 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:43.026189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ec449-0e57-49c7-af80-aa886f7eae83 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:43.026464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ec449-0e57-49c7-af80-aa886f7eae83 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:43.037692 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ec449-0e57-49c7-af80-aa886f7eae83 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:43.039506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ec449-0e57-49c7-af80-aa886f7eae83 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-de3ec449-0e57-49c7-af80-aa886f7eae83 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:43.057251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:43.058307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:43.059265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:43.059265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:43.059265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:43.059265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:43.059265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:43.059265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:43.059265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:43.059265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:43.059265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:58:43.059265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:43.063785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 491/979] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Aug 24 09:58:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:43.066138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ec449-0e57-49c7-af80-aa886f7eae83 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:43.091145 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:43.171742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af8bec-499c-4885-a73e-d0e8b8651a9d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:43.211591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ec449-0e57-49c7-af80-aa886f7eae83 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:43.212150 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52c1ab-f827-4c58-9bc3-8547c51c9d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:43.215471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52c1ab-f827-4c58-9bc3-8547c51c9d26 None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:43.215471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52c1ab-f827-4c58-9bc3-8547c51c9d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:43.215471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52c1ab-f827-4c58-9bc3-8547c51c9d26 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:43.217642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52c1ab-f827-4c58-9bc3-8547c51c9d26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:43.217642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52c1ab-f827-4c58-9bc3-8547c51c9d26 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:43.219989 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: WARNING keystone.server.flask.common [None req-9e52c1ab-f827-4c58-9bc3-8547c51c9d26 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 09:58:43.252792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3544b9c-c233-4c9e-a770-119763c8d605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:43.253439 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3544b9c-c233-4c9e-a770-119763c8d605 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:43.253700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3544b9c-c233-4c9e-a770-119763c8d605 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:43.253967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3544b9c-c233-4c9e-a770-119763c8d605 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:43.254699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3544b9c-c233-4c9e-a770-119763c8d605 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:43.254994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3544b9c-c233-4c9e-a770-119763c8d605 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:43.256413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3544b9c-c233-4c9e-a770-119763c8d605 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:43.271469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.sql.core [None req-9e52c1ab-f827-4c58-9bc3-8547c51c9d26 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:58:43.271469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 09:58:43.271469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [parameters: {'user_id': '367963b35b084d1aa99e495b9d4a04ec', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 09:58:43.271469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81930) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:58:43.276531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 492/980] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Aug 24 09:58:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:43.282184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3544b9c-c233-4c9e-a770-119763c8d605 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-9e52c1ab-f827-4c58-9bc3-8547c51c9d26 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application result.read() Aug 24 09:58:43.292883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:58:43.294088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application yield resource Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise self.value Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application session.commit() Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:58:43.295327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:58:43.296479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:58:43.303259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application result.read() Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:58:43.304752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [parameters: {'user_id': '367963b35b084d1aa99e495b9d4a04ec', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:43.305678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 09:58:43.306676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 09:58:43.307660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 09:58:43.307660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 09:58:43.307660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:58:43.307660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:58:43.307660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 09:58:43.307660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:43.307660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 489/981] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Aug 24 09:58:42 2022] POST /identity/v3/users => generated 160 bytes in 314 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:43.307660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52c1ab-f827-4c58-9bc3-8547c51c9d26 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:43.381765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:43.398024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:43.419722 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3544b9c-c233-4c9e-a770-119763c8d605 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:43.438901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52c1ab-f827-4c58-9bc3-8547c51c9d26 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:43.467162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb114da-9dc8-41f4-80f0-80c2cb8b52e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:43.467944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb114da-9dc8-41f4-80f0-80c2cb8b52e6 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:43.468230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb114da-9dc8-41f4-80f0-80c2cb8b52e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:43.468505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb114da-9dc8-41f4-80f0-80c2cb8b52e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:43.478512 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a2324-e3e5-45c4-88b6-c0630b1779b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:43.479579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a2324-e3e5-45c4-88b6-c0630b1779b9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:43.480092 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a2324-e3e5-45c4-88b6-c0630b1779b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:43.480583 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a2324-e3e5-45c4-88b6-c0630b1779b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:43.490012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a2324-e3e5-45c4-88b6-c0630b1779b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:43.492024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a2324-e3e5-45c4-88b6-c0630b1779b9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-df3a2324-e3e5-45c4-88b6-c0630b1779b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:43.504974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:43.505980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:43.506977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:43.506977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:43.506977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:43.506977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:43.506977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:43.506977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:43.506977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:43.506977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:43.506977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:58:43.506977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:43.513205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 490/982] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:58:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:43.516192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a2324-e3e5-45c4-88b6-c0630b1779b9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:43.555685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb114da-9dc8-41f4-80f0-80c2cb8b52e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:43.558817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb114da-9dc8-41f4-80f0-80c2cb8b52e6 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:43.630881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a2324-e3e5-45c4-88b6-c0630b1779b9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:43.666476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:43.670496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:43.677455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefa498-b6e1-4e27-bb04-383963fd7d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:43.686153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefa498-b6e1-4e27-bb04-383963fd7d7f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:43.686153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefa498-b6e1-4e27-bb04-383963fd7d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:43.686153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefa498-b6e1-4e27-bb04-383963fd7d7f None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:43.686153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceefa498-b6e1-4e27-bb04-383963fd7d7f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:43.686153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceefa498-b6e1-4e27-bb04-383963fd7d7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:43.686153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceefa498-b6e1-4e27-bb04-383963fd7d7f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:43.697757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 493/983] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:43 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:58:43.703267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb114da-9dc8-41f4-80f0-80c2cb8b52e6 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:43.726381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 491/984] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Aug 24 09:58:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:43.833957 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb114da-9dc8-41f4-80f0-80c2cb8b52e6 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:43.874791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92f109-8f98-46fd-9cf1-6626538c8c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:43.875495 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92f109-8f98-46fd-9cf1-6626538c8c66 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:43.875780 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92f109-8f98-46fd-9cf1-6626538c8c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:43.876053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92f109-8f98-46fd-9cf1-6626538c8c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:43.889399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92f109-8f98-46fd-9cf1-6626538c8c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:43.890848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92f109-8f98-46fd-9cf1-6626538c8c66 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-3f92f109-8f98-46fd-9cf1-6626538c8c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:43.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:43.907302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:43.909040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:43.909040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:43.909040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:43.909040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 494/985] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:43.914290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefa498-b6e1-4e27-bb04-383963fd7d7f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:44.027968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefa498-b6e1-4e27-bb04-383963fd7d7f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:44.065524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8060104-0d97-4ffd-8c1a-b46c5a097da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:44.066135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060104-0d97-4ffd-8c1a-b46c5a097da5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:44.066486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060104-0d97-4ffd-8c1a-b46c5a097da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:44.066740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060104-0d97-4ffd-8c1a-b46c5a097da5 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:44.067608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8060104-0d97-4ffd-8c1a-b46c5a097da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:44.068130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8060104-0d97-4ffd-8c1a-b46c5a097da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:44.069545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8060104-0d97-4ffd-8c1a-b46c5a097da5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:44.083911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 492/986] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:44.089868 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92f109-8f98-46fd-9cf1-6626538c8c66 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:44.120350 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:44.201424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92f109-8f98-46fd-9cf1-6626538c8c66 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:44.239798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b17bc-a9a5-4511-aa99-1101c2760c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:44.240434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b17bc-a9a5-4511-aa99-1101c2760c15 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:44.240739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b17bc-a9a5-4511-aa99-1101c2760c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:44.241041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b17bc-a9a5-4511-aa99-1101c2760c15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:44.247924 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b17bc-a9a5-4511-aa99-1101c2760c15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:44.249401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b17bc-a9a5-4511-aa99-1101c2760c15 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-7a1b17bc-a9a5-4511-aa99-1101c2760c15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:44.261657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:44.263024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:44.263949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:44.263949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:44.263949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:44.263949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:44.263949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:44.263949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:44.263949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:44.263949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:44.263949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 495/987] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:44.271318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8060104-0d97-4ffd-8c1a-b46c5a097da5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:44.326204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:44.383625 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8060104-0d97-4ffd-8c1a-b46c5a097da5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:44.422248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c42d49-336a-4c5f-98a5-f4a428761885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:44.422873 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c42d49-336a-4c5f-98a5-f4a428761885 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:44.423134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c42d49-336a-4c5f-98a5-f4a428761885 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:44.423387 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c42d49-336a-4c5f-98a5-f4a428761885 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:44.424368 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c42d49-336a-4c5f-98a5-f4a428761885 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:44.424800 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c42d49-336a-4c5f-98a5-f4a428761885 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:44.426361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c42d49-336a-4c5f-98a5-f4a428761885 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:44.439071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 493/988] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:44.446051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b17bc-a9a5-4511-aa99-1101c2760c15 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:44.586851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:44.602213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b17bc-a9a5-4511-aa99-1101c2760c15 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:44.638086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad75e25-9cec-480f-9065-bf1b8c102532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:44.638607 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75e25-9cec-480f-9065-bf1b8c102532 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:44.638882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75e25-9cec-480f-9065-bf1b8c102532 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:44.639149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75e25-9cec-480f-9065-bf1b8c102532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:44.639861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad75e25-9cec-480f-9065-bf1b8c102532 None admin] RBAC: Adding query filter params (role.id=47c08bf673724b40aa00a201adb1411b, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=8b7227998e7846c7a9bc14b12b637ee0) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:44.640161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad75e25-9cec-480f-9065-bf1b8c102532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:44.641601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad75e25-9cec-480f-9065-bf1b8c102532 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:44.648793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 496/989] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:44 2022] GET /identity/v3/role_assignments?user.id=8b7227998e7846c7a9bc14b12b637ee0&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=47c08bf673724b40aa00a201adb1411b => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:44.675632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e037dcd-8479-437e-a361-3b366e550b93 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:44.675953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e037dcd-8479-437e-a361-3b366e550b93 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:44.676196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e037dcd-8479-437e-a361-3b366e550b93 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:44.677826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 494/990] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:44.683630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2beb21d-cecc-4f0b-9c13-084195ac572c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:44.683942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2beb21d-cecc-4f0b-9c13-084195ac572c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:44.684189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2beb21d-cecc-4f0b-9c13-084195ac572c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:44.704524 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-f2beb21d-cecc-4f0b-9c13-084195ac572c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:44.706774 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f2beb21d-cecc-4f0b-9c13-084195ac572c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:44.790120 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 497/991] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:45.054529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd23ec-db14-4a38-8121-dc81682af259 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:45.054721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd23ec-db14-4a38-8121-dc81682af259 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:45.054984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd23ec-db14-4a38-8121-dc81682af259 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:45.056527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 495/992] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:45.063245 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2beb21d-cecc-4f0b-9c13-084195ac572c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:45.066798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d52783-f132-49aa-a8a8-a925e341e05e None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:45.067080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d52783-f132-49aa-a8a8-a925e341e05e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:45.067432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d52783-f132-49aa-a8a8-a925e341e05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:45.088933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b0d52783-f132-49aa-a8a8-a925e341e05e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:45.090493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b0d52783-f132-49aa-a8a8-a925e341e05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:45.167555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 496/993] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:45.186621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:45.214967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2beb21d-cecc-4f0b-9c13-084195ac572c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:45.273002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8215251-e0b2-4aec-8de0-1617e1c9e58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:45.273002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8215251-e0b2-4aec-8de0-1617e1c9e58d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:45.273002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8215251-e0b2-4aec-8de0-1617e1c9e58d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:45.273002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8215251-e0b2-4aec-8de0-1617e1c9e58d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:45.273847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8215251-e0b2-4aec-8de0-1617e1c9e58d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:45.273847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8215251-e0b2-4aec-8de0-1617e1c9e58d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-b8215251-e0b2-4aec-8de0-1617e1c9e58d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:45.282698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:45.284288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:45.286228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 498/994] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Aug 24 09:58:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:45.291038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d52783-f132-49aa-a8a8-a925e341e05e None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:45.350107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:45.442627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8215251-e0b2-4aec-8de0-1617e1c9e58d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:45.465705 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d52783-f132-49aa-a8a8-a925e341e05e None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:45.513313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7088c-cb1c-4100-a74a-187e14c014dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:45.513947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7088c-cb1c-4100-a74a-187e14c014dd None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:45.514227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7088c-cb1c-4100-a74a-187e14c014dd None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:45.514506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7088c-cb1c-4100-a74a-187e14c014dd None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:45.515657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7088c-cb1c-4100-a74a-187e14c014dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:45.515969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7088c-cb1c-4100-a74a-187e14c014dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:45.517923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7088c-cb1c-4100-a74a-187e14c014dd None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:45.523309 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 497/995] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Aug 24 09:58:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:45.529043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7088c-cb1c-4100-a74a-187e14c014dd None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:45.586212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8215251-e0b2-4aec-8de0-1617e1c9e58d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:45.628133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79ac9f-d16f-4871-8adf-eb493735699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:45.628687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79ac9f-d16f-4871-8adf-eb493735699e None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:45.629020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79ac9f-d16f-4871-8adf-eb493735699e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:45.629310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79ac9f-d16f-4871-8adf-eb493735699e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:45.630859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79ac9f-d16f-4871-8adf-eb493735699e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:45.632633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79ac9f-d16f-4871-8adf-eb493735699e None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-4b79ac9f-d16f-4871-8adf-eb493735699e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:45.643151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:45.644367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:45.646208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 499/996] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:45.650162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79ac9f-d16f-4871-8adf-eb493735699e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:45.655833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:45.660839 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7088c-cb1c-4100-a74a-187e14c014dd None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:45.666170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:45.702602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88520c-0e8a-4f2a-a623-d720f1c1e96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:45.703219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88520c-0e8a-4f2a-a623-d720f1c1e96e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:45.703545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88520c-0e8a-4f2a-a623-d720f1c1e96e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:45.703959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88520c-0e8a-4f2a-a623-d720f1c1e96e None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:45.705270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88520c-0e8a-4f2a-a623-d720f1c1e96e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:45.705805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88520c-0e8a-4f2a-a623-d720f1c1e96e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:45.707491 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88520c-0e8a-4f2a-a623-d720f1c1e96e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:45.713745 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 498/997] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Aug 24 09:58:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:45.719557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88520c-0e8a-4f2a-a623-d720f1c1e96e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:45.778704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79ac9f-d16f-4871-8adf-eb493735699e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:45.822456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50e5a7-20c9-4edf-92a9-8fc423f3d7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:45.823691 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50e5a7-20c9-4edf-92a9-8fc423f3d7fe None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:45.824217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50e5a7-20c9-4edf-92a9-8fc423f3d7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:45.824985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50e5a7-20c9-4edf-92a9-8fc423f3d7fe None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:45.826360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50e5a7-20c9-4edf-92a9-8fc423f3d7fe None admin] RBAC: Adding query filter params (name=compute) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:45.827110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50e5a7-20c9-4edf-92a9-8fc423f3d7fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:45.829422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50e5a7-20c9-4edf-92a9-8fc423f3d7fe None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:45.835166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88520c-0e8a-4f2a-a623-d720f1c1e96e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:45.837036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 500/998] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:45.843425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50e5a7-20c9-4edf-92a9-8fc423f3d7fe None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:45.880545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bd03e-16be-4397-8da5-a2300a8db4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:45.881228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bd03e-16be-4397-8da5-a2300a8db4a9 None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:45.881581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bd03e-16be-4397-8da5-a2300a8db4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:45.881931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bd03e-16be-4397-8da5-a2300a8db4a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:45.882619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bd03e-16be-4397-8da5-a2300a8db4a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:45.884000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bd03e-16be-4397-8da5-a2300a8db4a9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:45.930533 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:45.961728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50e5a7-20c9-4edf-92a9-8fc423f3d7fe None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:45.995590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fde19b-0365-45ee-ab7c-6a41e7603697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:46.002251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fde19b-0365-45ee-ab7c-6a41e7603697 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:46.002251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fde19b-0365-45ee-ab7c-6a41e7603697 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:46.002251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fde19b-0365-45ee-ab7c-6a41e7603697 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:46.002251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fde19b-0365-45ee-ab7c-6a41e7603697 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:46.002251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fde19b-0365-45ee-ab7c-6a41e7603697 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:46.002251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fde19b-0365-45ee-ab7c-6a41e7603697 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:46.005541 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 501/999] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:46.009771 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 499/1000] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 09:58:45 2022] POST /identity/v3/endpoints => generated 325 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:58:46.011522 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fde19b-0365-45ee-ab7c-6a41e7603697 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:46.145896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fde19b-0365-45ee-ab7c-6a41e7603697 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:46.185560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:46.191437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb29b58-001b-400f-9127-92a03e87d022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:46.191868 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb29b58-001b-400f-9127-92a03e87d022 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:46.192140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb29b58-001b-400f-9127-92a03e87d022 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:46.192403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb29b58-001b-400f-9127-92a03e87d022 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:46.193172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb29b58-001b-400f-9127-92a03e87d022 None admin] RBAC: Adding query filter params (interface=public, service_id=9b4ac416456f444e85b8d86e5b57f4e0, region_id=RegionOne) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:46.193448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb29b58-001b-400f-9127-92a03e87d022 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:46.194860 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb29b58-001b-400f-9127-92a03e87d022 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:46.200161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 502/1001] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Aug 24 09:58:46 2022] GET /identity/v3/endpoints?service_id=9b4ac416456f444e85b8d86e5b57f4e0&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:46.206384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bd03e-16be-4397-8da5-a2300a8db4a9 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:46.237503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:46.359621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bd03e-16be-4397-8da5-a2300a8db4a9 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:46.396895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f734286-03e3-4e1a-aa0e-97ba942bc5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:46.397445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f734286-03e3-4e1a-aa0e-97ba942bc5ca None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:46.397727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f734286-03e3-4e1a-aa0e-97ba942bc5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:46.398003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f734286-03e3-4e1a-aa0e-97ba942bc5ca None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:46.398680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f734286-03e3-4e1a-aa0e-97ba942bc5ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:46.400063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f734286-03e3-4e1a-aa0e-97ba942bc5ca None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:46.412910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 500/1002] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Aug 24 09:58:46 2022] GET /identity/v3/services => generated 1727 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:58:46.683299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac2e60-8e70-4908-8034-7e1855e232ae None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:46.683953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac2e60-8e70-4908-8034-7e1855e232ae None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:46.683953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac2e60-8e70-4908-8034-7e1855e232ae None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:46.685325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 503/1003] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:46.690746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514ac86-dbd0-40a7-b797-9da998e4f7ac None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:46.691193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514ac86-dbd0-40a7-b797-9da998e4f7ac None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:46.691498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514ac86-dbd0-40a7-b797-9da998e4f7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:46.711510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-2514ac86-dbd0-40a7-b797-9da998e4f7ac None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:46.712811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2514ac86-dbd0-40a7-b797-9da998e4f7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:46.788902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 501/1004] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:47.088785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac2e60-8e70-4908-8034-7e1855e232ae None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:47.207176 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac2e60-8e70-4908-8034-7e1855e232ae None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:47.245693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d242589-200b-494e-84a1-1f9c751ae807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:47.246314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d242589-200b-494e-84a1-1f9c751ae807 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:47.246589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d242589-200b-494e-84a1-1f9c751ae807 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:47.246959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d242589-200b-494e-84a1-1f9c751ae807 None admin] PATH_INFO: `/v3/services/image` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:47.247530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d242589-200b-494e-84a1-1f9c751ae807 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:47.248902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d242589-200b-494e-84a1-1f9c751ae807 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-6d242589-200b-494e-84a1-1f9c751ae807 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:47.257576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:47.258776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:47.259624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 504/1005] 10.222.0.30 () {56 vars in 1105 bytes} [Wed Aug 24 09:58:47 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:47.264702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514ac86-dbd0-40a7-b797-9da998e4f7ac None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:47.284653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:47.380414 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514ac86-dbd0-40a7-b797-9da998e4f7ac None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:47.426070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cce3a1-5425-4eae-9daf-295c3d34f519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:47.426665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cce3a1-5425-4eae-9daf-295c3d34f519 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:47.426941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cce3a1-5425-4eae-9daf-295c3d34f519 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:47.427209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cce3a1-5425-4eae-9daf-295c3d34f519 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:47.427837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cce3a1-5425-4eae-9daf-295c3d34f519 None admin] RBAC: Adding query filter params (name=image) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:47.428135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cce3a1-5425-4eae-9daf-295c3d34f519 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:47.429524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cce3a1-5425-4eae-9daf-295c3d34f519 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:47.439995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 502/1006] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:47.443900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d242589-200b-494e-84a1-1f9c751ae807 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:47.507509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:47.554067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d242589-200b-494e-84a1-1f9c751ae807 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:47.589300 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08687dc1-5c3b-42b3-9819-f9c3eadcca3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:47.589908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08687dc1-5c3b-42b3-9819-f9c3eadcca3c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:47.590188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08687dc1-5c3b-42b3-9819-f9c3eadcca3c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:47.590463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08687dc1-5c3b-42b3-9819-f9c3eadcca3c None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:47.591091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08687dc1-5c3b-42b3-9819-f9c3eadcca3c None admin] RBAC: Adding query filter params (type=image) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:47.591391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08687dc1-5c3b-42b3-9819-f9c3eadcca3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:47.592789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08687dc1-5c3b-42b3-9819-f9c3eadcca3c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:47.598477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 505/1007] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:47.997465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dc728-214a-493c-b4f7-f019b9a2a9c4 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:47.998015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dc728-214a-493c-b4f7-f019b9a2a9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:47.998015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dc728-214a-493c-b4f7-f019b9a2a9c4 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:47.999396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 503/1008] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:48.005211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78331b-4771-4acf-9a83-82d42175d626 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:48.005726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78331b-4771-4acf-9a83-82d42175d626 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:48.005824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78331b-4771-4acf-9a83-82d42175d626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:48.025976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-0e78331b-4771-4acf-9a83-82d42175d626 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:48.027352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0e78331b-4771-4acf-9a83-82d42175d626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:48.110777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 506/1009] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:48 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:48.390169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dc728-214a-493c-b4f7-f019b9a2a9c4 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:48.404646 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e115-2cdf-4ed0-8b35-9a68469ed8ac None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:48.405015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e115-2cdf-4ed0-8b35-9a68469ed8ac None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:48.405202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e115-2cdf-4ed0-8b35-9a68469ed8ac None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:48.406653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 507/1010] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:58:48.412634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4e08f-65cd-45a7-9784-92b7bcbc58ce None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:48.412954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4e08f-65cd-45a7-9784-92b7bcbc58ce None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:48.413240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4e08f-65cd-45a7-9784-92b7bcbc58ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:48.435232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e0f4e08f-65cd-45a7-9784-92b7bcbc58ce None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:48.437484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e0f4e08f-65cd-45a7-9784-92b7bcbc58ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:48.494711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:48.500115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:48.516786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dc728-214a-493c-b4f7-f019b9a2a9c4 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:48.520918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 508/1011] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:48 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:48.568647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba72e5-d0a3-4baf-85b2-4da858b551c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:48.569431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba72e5-d0a3-4baf-85b2-4da858b551c6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:48.569431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba72e5-d0a3-4baf-85b2-4da858b551c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:48.569798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba72e5-d0a3-4baf-85b2-4da858b551c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:48.570844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba72e5-d0a3-4baf-85b2-4da858b551c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:48.572404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba72e5-d0a3-4baf-85b2-4da858b551c6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:48.584884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 504/1012] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:48.585097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4e08f-65cd-45a7-9784-92b7bcbc58ce None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:48.683055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4e08f-65cd-45a7-9784-92b7bcbc58ce None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:48.721016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385413cd-c346-4504-bce5-25621012d5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:48.721520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385413cd-c346-4504-bce5-25621012d5d9 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:48.721796 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385413cd-c346-4504-bce5-25621012d5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:48.722065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385413cd-c346-4504-bce5-25621012d5d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:48.722699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385413cd-c346-4504-bce5-25621012d5d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:48.724111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385413cd-c346-4504-bce5-25621012d5d9 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:48.796782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba72e5-d0a3-4baf-85b2-4da858b551c6 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:48.813268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 509/1013] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:58:48 2022] POST /identity/v3/projects => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:58:48.915893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba72e5-d0a3-4baf-85b2-4da858b551c6 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:48.955977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a31473-089f-46e0-8492-2bf65b1c073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:48.957760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a31473-089f-46e0-8492-2bf65b1c073b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:48.957760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a31473-089f-46e0-8492-2bf65b1c073b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:48.957760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a31473-089f-46e0-8492-2bf65b1c073b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:48.957760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a31473-089f-46e0-8492-2bf65b1c073b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:48.959162 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a31473-089f-46e0-8492-2bf65b1c073b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-f6a31473-089f-46e0-8492-2bf65b1c073b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:48.969866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:48.971468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:48.972303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 505/1014] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:48.977593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385413cd-c346-4504-bce5-25621012d5d9 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:49.016682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:49.087063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385413cd-c346-4504-bce5-25621012d5d9 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:49.125005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca2381-7191-422b-8b10-f4010d36ddba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:49.125605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca2381-7191-422b-8b10-f4010d36ddba None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:49.125883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca2381-7191-422b-8b10-f4010d36ddba None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:49.126280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca2381-7191-422b-8b10-f4010d36ddba None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:49.127167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca2381-7191-422b-8b10-f4010d36ddba None admin] RBAC: Adding query filter params (name=compute) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:49.127470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca2381-7191-422b-8b10-f4010d36ddba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:49.128846 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca2381-7191-422b-8b10-f4010d36ddba None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:49.134413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 510/1015] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:49.139971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a31473-089f-46e0-8492-2bf65b1c073b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:49.156398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:49.252171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a31473-089f-46e0-8492-2bf65b1c073b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:49.300519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fd599-0432-45f0-b4cb-8edebaf33f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:49.301537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fd599-0432-45f0-b4cb-8edebaf33f2b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:49.301537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fd599-0432-45f0-b4cb-8edebaf33f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:49.301851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fd599-0432-45f0-b4cb-8edebaf33f2b None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:49.302463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fd599-0432-45f0-b4cb-8edebaf33f2b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:49.303189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fd599-0432-45f0-b4cb-8edebaf33f2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:49.304185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fd599-0432-45f0-b4cb-8edebaf33f2b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:49.315099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 506/1016] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Aug 24 09:58:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:49.318406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca2381-7191-422b-8b10-f4010d36ddba None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:49.439769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:49.443518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca2381-7191-422b-8b10-f4010d36ddba None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:49.485202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57b42b-1f1e-4625-b14f-8adbe3a6c90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:49.486465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57b42b-1f1e-4625-b14f-8adbe3a6c90d None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:49.487149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57b42b-1f1e-4625-b14f-8adbe3a6c90d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:49.487626 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57b42b-1f1e-4625-b14f-8adbe3a6c90d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:49.488565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57b42b-1f1e-4625-b14f-8adbe3a6c90d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:49.490159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57b42b-1f1e-4625-b14f-8adbe3a6c90d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:49.568813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c502634-6201-4eab-ab06-b48f07585b89 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:49.569061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c502634-6201-4eab-ab06-b48f07585b89 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:49.569364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c502634-6201-4eab-ab06-b48f07585b89 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:49.570729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 507/1017] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:58:49.575747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4d324-f85e-46bc-830e-1537af8f0c24 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:49.576032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4d324-f85e-46bc-830e-1537af8f0c24 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:49.576309 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4d324-f85e-46bc-830e-1537af8f0c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:49.595884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-5cd4d324-f85e-46bc-830e-1537af8f0c24 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:49.597055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5cd4d324-f85e-46bc-830e-1537af8f0c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:49.597827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 511/1018] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 09:58:49 2022] POST /identity/v3/endpoints => generated 334 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:49.698639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 508/1019] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:49 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:49.973580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57b42b-1f1e-4625-b14f-8adbe3a6c90d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:50.116060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57b42b-1f1e-4625-b14f-8adbe3a6c90d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:50.154518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82075a92-82a2-4807-8a7b-f093c5ff6e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:50.155236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82075a92-82a2-4807-8a7b-f093c5ff6e83 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:50.155628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82075a92-82a2-4807-8a7b-f093c5ff6e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:50.155966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82075a92-82a2-4807-8a7b-f093c5ff6e83 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:50.156721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82075a92-82a2-4807-8a7b-f093c5ff6e83 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:50.158276 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82075a92-82a2-4807-8a7b-f093c5ff6e83 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:50.238401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 512/1020] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:58:49 2022] POST /identity/v3/services => generated 242 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:50.936038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d267531-f406-4819-bcac-bba4409a386b None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:50.936585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d267531-f406-4819-bcac-bba4409a386b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:50.936585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d267531-f406-4819-bcac-bba4409a386b None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:50.938134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 509/1021] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:58:50.944480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566d6a5-3f48-43ae-b5de-b60f9852080c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:50.944767 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566d6a5-3f48-43ae-b5de-b60f9852080c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:50.945080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566d6a5-3f48-43ae-b5de-b60f9852080c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:50.974250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-7566d6a5-3f48-43ae-b5de-b60f9852080c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:50.975456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7566d6a5-3f48-43ae-b5de-b60f9852080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:51.012142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:51.069830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 513/1022] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 24 09:58:51.356722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d267531-f406-4819-bcac-bba4409a386b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:51.359900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:51.491894 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d267531-f406-4819-bcac-bba4409a386b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:51.530167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2f283-b1f1-402d-a071-7d1a9df6d6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:51.530969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2f283-b1f1-402d-a071-7d1a9df6d6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:51.531436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2f283-b1f1-402d-a071-7d1a9df6d6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:51.531860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2f283-b1f1-402d-a071-7d1a9df6d6e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:51.533706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2f283-b1f1-402d-a071-7d1a9df6d6e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:51.535584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2f283-b1f1-402d-a071-7d1a9df6d6e2 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:51.540900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 510/1023] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:51.542861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566d6a5-3f48-43ae-b5de-b60f9852080c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:51.650452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566d6a5-3f48-43ae-b5de-b60f9852080c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:51.690805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6ff6a-b020-451b-86c6-98d3cdb38610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:51.691413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ff6a-b020-451b-86c6-98d3cdb38610 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:51.691704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ff6a-b020-451b-86c6-98d3cdb38610 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:51.691980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ff6a-b020-451b-86c6-98d3cdb38610 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:51.692637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6ff6a-b020-451b-86c6-98d3cdb38610 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:51.694109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6ff6a-b020-451b-86c6-98d3cdb38610 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:51.747684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfdfd0-534e-468e-9736-be1047e5ea00 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:51.747977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfdfd0-534e-468e-9736-be1047e5ea00 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:51.749143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfdfd0-534e-468e-9736-be1047e5ea00 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:51.749932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 511/1024] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:51.757928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287460d-c8c3-493d-af60-0bf006a629b8 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:51.758224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287460d-c8c3-493d-af60-0bf006a629b8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:51.758507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287460d-c8c3-493d-af60-0bf006a629b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:51.777660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-5287460d-c8c3-493d-af60-0bf006a629b8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:51.778871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5287460d-c8c3-493d-af60-0bf006a629b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:51.813721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:51.822753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 514/1025] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:58:51 2022] POST /identity/v3/users => generated 285 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:58:51.861425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:51.868901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 512/1026] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:52.151183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6ff6a-b020-451b-86c6-98d3cdb38610 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:52.246170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdd6e9-a860-48a5-83ab-2640281650dc None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:52.246719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdd6e9-a860-48a5-83ab-2640281650dc None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:52.246719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdd6e9-a860-48a5-83ab-2640281650dc None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:52.248050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 513/1027] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:52.253628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de36760-022e-4fcd-9b07-87a0574d75a2 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:52.253911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de36760-022e-4fcd-9b07-87a0574d75a2 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:52.254185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de36760-022e-4fcd-9b07-87a0574d75a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:52.262658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6ff6a-b020-451b-86c6-98d3cdb38610 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:52.272903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3de36760-022e-4fcd-9b07-87a0574d75a2 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:52.274104 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3de36760-022e-4fcd-9b07-87a0574d75a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:52.306903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2499c-c5e4-485b-a117-428a8ec61c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:52.307536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2499c-c5e4-485b-a117-428a8ec61c8b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:52.307812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2499c-c5e4-485b-a117-428a8ec61c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:52.308085 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2499c-c5e4-485b-a117-428a8ec61c8b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:52.319467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2499c-c5e4-485b-a117-428a8ec61c8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-d6b2499c-c5e4-485b-a117-428a8ec61c8b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:52.324787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:52.327223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:52.328162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 515/1028] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Aug 24 09:58:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:52.333073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2499c-c5e4-485b-a117-428a8ec61c8b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:52.348919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 514/1029] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:52.397904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:52.447774 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2499c-c5e4-485b-a117-428a8ec61c8b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:52.489151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99759e-3809-461b-8bcf-dfacf3c1cfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:52.489698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99759e-3809-461b-8bcf-dfacf3c1cfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:52.489978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99759e-3809-461b-8bcf-dfacf3c1cfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:52.490527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99759e-3809-461b-8bcf-dfacf3c1cfa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:52.491308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99759e-3809-461b-8bcf-dfacf3c1cfa4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:52.491308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99759e-3809-461b-8bcf-dfacf3c1cfa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:52.492584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99759e-3809-461b-8bcf-dfacf3c1cfa4 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:52.510304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 516/1030] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Aug 24 09:58:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:52.512363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de36760-022e-4fcd-9b07-87a0574d75a2 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:52.622932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99759e-3809-461b-8bcf-dfacf3c1cfa4 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:52.630720 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de36760-022e-4fcd-9b07-87a0574d75a2 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:52.674523 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:52.683873 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09866511-e922-4570-b5bc-1400ddffdde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:52.684527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09866511-e922-4570-b5bc-1400ddffdde2 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:52.684809 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09866511-e922-4570-b5bc-1400ddffdde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:52.685113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09866511-e922-4570-b5bc-1400ddffdde2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:52.698219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09866511-e922-4570-b5bc-1400ddffdde2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:52.699625 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09866511-e922-4570-b5bc-1400ddffdde2 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-09866511-e922-4570-b5bc-1400ddffdde2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:52.716662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:52.717883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:52.719275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:52.719275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:52.719275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:52.719275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 515/1031] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:52.723936 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09866511-e922-4570-b5bc-1400ddffdde2 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:52.768709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99759e-3809-461b-8bcf-dfacf3c1cfa4 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:52.792206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:52.812178 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf239e-b607-4e1d-ad7e-b928ffde1b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:52.813041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf239e-b607-4e1d-ad7e-b928ffde1b67 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:52.813557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf239e-b607-4e1d-ad7e-b928ffde1b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:52.814430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf239e-b607-4e1d-ad7e-b928ffde1b67 None admin] PATH_INFO: `/v3/services/image` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:52.815023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf239e-b607-4e1d-ad7e-b928ffde1b67 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:52.816430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf239e-b607-4e1d-ad7e-b928ffde1b67 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-70bf239e-b607-4e1d-ad7e-b928ffde1b67 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:52.825697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:52.826961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:52.827890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 517/1032] 10.222.0.30 () {56 vars in 1105 bytes} [Wed Aug 24 09:58:52 2022] GET /identity/v3/services/image => generated 86 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:52.834737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf239e-b607-4e1d-ad7e-b928ffde1b67 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:52.862371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09866511-e922-4570-b5bc-1400ddffdde2 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:52.906417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399da297-2f15-4a9f-9853-2be46bf29ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:52.907272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399da297-2f15-4a9f-9853-2be46bf29ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:52.907733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399da297-2f15-4a9f-9853-2be46bf29ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:52.908170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399da297-2f15-4a9f-9853-2be46bf29ec0 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:52.909209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399da297-2f15-4a9f-9853-2be46bf29ec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:52.910083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399da297-2f15-4a9f-9853-2be46bf29ec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:52.911834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399da297-2f15-4a9f-9853-2be46bf29ec0 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:52.926334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 516/1033] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:52.931516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399da297-2f15-4a9f-9853-2be46bf29ec0 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:52.958964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf239e-b607-4e1d-ad7e-b928ffde1b67 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:52.985193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:52.997515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a2694-97d2-4408-8bc3-2a15b6eb619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:53.011134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a2694-97d2-4408-8bc3-2a15b6eb619e None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:53.011134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a2694-97d2-4408-8bc3-2a15b6eb619e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:53.011134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a2694-97d2-4408-8bc3-2a15b6eb619e None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:53.011134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a2694-97d2-4408-8bc3-2a15b6eb619e None admin] RBAC: Adding query filter params (name=image) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:53.011134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a2694-97d2-4408-8bc3-2a15b6eb619e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:53.011134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a2694-97d2-4408-8bc3-2a15b6eb619e None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:53.011659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 518/1034] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:53.012144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a2694-97d2-4408-8bc3-2a15b6eb619e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:53.051904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399da297-2f15-4a9f-9853-2be46bf29ec0 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:53.067552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:53.091579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c27423-9512-4184-8ca9-92ff206d7979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:53.092210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c27423-9512-4184-8ca9-92ff206d7979 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:53.092491 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c27423-9512-4184-8ca9-92ff206d7979 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:53.092764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c27423-9512-4184-8ca9-92ff206d7979 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:53.101186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c27423-9512-4184-8ca9-92ff206d7979 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:53.102700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c27423-9512-4184-8ca9-92ff206d7979 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b1c27423-9512-4184-8ca9-92ff206d7979 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:53.116029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:53.117143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:53.118106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:53.118106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:53.118106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:53.118106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:53.118106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:53.118106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:53.118106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:53.118106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:53.118106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:53.118106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:53.118106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 517/1035] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Aug 24 09:58:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:53.122834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c27423-9512-4184-8ca9-92ff206d7979 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:53.125331 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a2694-97d2-4408-8bc3-2a15b6eb619e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:53.164021 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d5262-7490-4ba5-84e2-212097f4d5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:53.164976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d5262-7490-4ba5-84e2-212097f4d5ba None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:53.165454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d5262-7490-4ba5-84e2-212097f4d5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:53.165914 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d5262-7490-4ba5-84e2-212097f4d5ba None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:53.166724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d5262-7490-4ba5-84e2-212097f4d5ba None admin] RBAC: Adding query filter params (type=image) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:53.167226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d5262-7490-4ba5-84e2-212097f4d5ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:53.168772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d5262-7490-4ba5-84e2-212097f4d5ba None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:53.174577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 519/1036] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:53.180347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d5262-7490-4ba5-84e2-212097f4d5ba None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:53.235176 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c27423-9512-4184-8ca9-92ff206d7979 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:53.237472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:53.274186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874c850-5ccf-4143-ad2f-9279b484b487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:53.274866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c850-5ccf-4143-ad2f-9279b484b487 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:53.275098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c850-5ccf-4143-ad2f-9279b484b487 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:53.275665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c850-5ccf-4143-ad2f-9279b484b487 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:53.276660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874c850-5ccf-4143-ad2f-9279b484b487 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:53.277385 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874c850-5ccf-4143-ad2f-9279b484b487 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:53.278863 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874c850-5ccf-4143-ad2f-9279b484b487 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:53.302820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 518/1037] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Aug 24 09:58:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:53.302999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d5262-7490-4ba5-84e2-212097f4d5ba None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:53.305612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874c850-5ccf-4143-ad2f-9279b484b487 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:53.317864 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:53.342893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f39685-cd38-474e-a59a-4062df2f4f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:53.343518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f39685-cd38-474e-a59a-4062df2f4f91 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:53.343908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f39685-cd38-474e-a59a-4062df2f4f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:53.344077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f39685-cd38-474e-a59a-4062df2f4f91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:53.344777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f39685-cd38-474e-a59a-4062df2f4f91 None admin] RBAC: Adding query filter params (interface=public, service_id=92837a6428fb48618cbeab2742f9ff01, region_id=RegionOne) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:53.345202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f39685-cd38-474e-a59a-4062df2f4f91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:53.346792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f39685-cd38-474e-a59a-4062df2f4f91 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:53.352838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 520/1038] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Aug 24 09:58:53 2022] GET /identity/v3/endpoints?service_id=92837a6428fb48618cbeab2742f9ff01&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:53.358424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f39685-cd38-474e-a59a-4062df2f4f91 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:53.430229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874c850-5ccf-4143-ad2f-9279b484b487 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:53.460274 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f39685-cd38-474e-a59a-4062df2f4f91 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:53.470739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d72612-9f05-4869-aa0c-ce26240c240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:53.471356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d72612-9f05-4869-aa0c-ce26240c240e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:53.471669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d72612-9f05-4869-aa0c-ce26240c240e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:53.472025 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d72612-9f05-4869-aa0c-ce26240c240e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:53.495312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b9b99-264b-4923-a63d-bdebfdf70abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:53.495841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b9b99-264b-4923-a63d-bdebfdf70abd None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:53.496122 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b9b99-264b-4923-a63d-bdebfdf70abd None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:53.496398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b9b99-264b-4923-a63d-bdebfdf70abd None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:53.497044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b9b99-264b-4923-a63d-bdebfdf70abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:53.498407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b9b99-264b-4923-a63d-bdebfdf70abd None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:53.502801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:53.505619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 521/1039] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Aug 24 09:58:53 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:58:53.554138 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d72612-9f05-4869-aa0c-ce26240c240e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:53.555677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d72612-9f05-4869-aa0c-ce26240c240e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:53.668664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 519/1040] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:53 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:58:53.674745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b9b99-264b-4923-a63d-bdebfdf70abd None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:53.720762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:53.780583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b9b99-264b-4923-a63d-bdebfdf70abd None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:53.817147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829dec7-da02-4866-bb1a-b58956095876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:53.817779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829dec7-da02-4866-bb1a-b58956095876 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:53.818058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829dec7-da02-4866-bb1a-b58956095876 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:53.818333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829dec7-da02-4866-bb1a-b58956095876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:53.828219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b9a26-d314-4cc3-9c9e-c4db5ea1bb04 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:53.828508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b9a26-d314-4cc3-9c9e-c4db5ea1bb04 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:53.828778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b9a26-d314-4cc3-9c9e-c4db5ea1bb04 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:53.829502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829dec7-da02-4866-bb1a-b58956095876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:53.830795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 520/1041] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:53.831104 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829dec7-da02-4866-bb1a-b58956095876 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:53.835665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5699c0-d848-43c9-aed1-a9d83dfcb1db None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:53.835949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5699c0-d848-43c9-aed1-a9d83dfcb1db None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:53.836224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5699c0-d848-43c9-aed1-a9d83dfcb1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-5829dec7-da02-4866-bb1a-b58956095876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:53.846489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:53.847563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:53.848484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:53.848484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:53.848484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:53.852453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 522/1042] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:53.853211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829dec7-da02-4866-bb1a-b58956095876 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:53.854087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ce5699c0-d848-43c9-aed1-a9d83dfcb1db None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:53.855251 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ce5699c0-d848-43c9-aed1-a9d83dfcb1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:53.915085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:53.928557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 521/1043] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:53.949919 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:53.959091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829dec7-da02-4866-bb1a-b58956095876 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:53.995336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c49a7d-f66f-472b-92ba-f52b19af408a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:53.996288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c49a7d-f66f-472b-92ba-f52b19af408a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:53.996825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c49a7d-f66f-472b-92ba-f52b19af408a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:53.997351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c49a7d-f66f-472b-92ba-f52b19af408a None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:53.998199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c49a7d-f66f-472b-92ba-f52b19af408a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:53.998715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c49a7d-f66f-472b-92ba-f52b19af408a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:54.000314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c49a7d-f66f-472b-92ba-f52b19af408a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:54.013924 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 523/1044] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:54.019400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5699c0-d848-43c9-aed1-a9d83dfcb1db None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:54.132571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5699c0-d848-43c9-aed1-a9d83dfcb1db None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:54.167584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e204e-82ca-4e6d-a608-7aae046b6b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:54.168217 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e204e-82ca-4e6d-a608-7aae046b6b99 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:54.168498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e204e-82ca-4e6d-a608-7aae046b6b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:54.168773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e204e-82ca-4e6d-a608-7aae046b6b99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:54.176109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e204e-82ca-4e6d-a608-7aae046b6b99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:54.177617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e204e-82ca-4e6d-a608-7aae046b6b99 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-0b0e204e-82ca-4e6d-a608-7aae046b6b99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:54.188647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:54.189640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:54.191009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:54.191009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:54.191009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:54.191009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:54.191009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:54.191009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:54.191009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:54.191009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:54.191009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 522/1045] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:54.195687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c49a7d-f66f-472b-92ba-f52b19af408a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:54.197810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e204e-82ca-4e6d-a608-7aae046b6b99 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:54.252796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:54.308888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e204e-82ca-4e6d-a608-7aae046b6b99 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:54.315540 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c49a7d-f66f-472b-92ba-f52b19af408a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:54.344162 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059a786-ac4e-4b7a-a152-dc91b816072f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:54.344803 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059a786-ac4e-4b7a-a152-dc91b816072f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:54.345709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059a786-ac4e-4b7a-a152-dc91b816072f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:54.346489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059a786-ac4e-4b7a-a152-dc91b816072f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:54.351602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c02b2-5b12-4aff-834a-8877b1dbc11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:54.352197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c02b2-5b12-4aff-834a-8877b1dbc11a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:54.352472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c02b2-5b12-4aff-834a-8877b1dbc11a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:54.352742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c02b2-5b12-4aff-834a-8877b1dbc11a None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:54.353454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c02b2-5b12-4aff-834a-8877b1dbc11a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:54.353755 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c02b2-5b12-4aff-834a-8877b1dbc11a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:54.355158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c02b2-5b12-4aff-834a-8877b1dbc11a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:54.357644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059a786-ac4e-4b7a-a152-dc91b816072f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-d059a786-ac4e-4b7a-a152-dc91b816072f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:54.363727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:54.364718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:54.365611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 523/1046] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:54.370420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 524/1047] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:54.370708 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059a786-ac4e-4b7a-a152-dc91b816072f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:54.376788 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c02b2-5b12-4aff-834a-8877b1dbc11a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:54.394936 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:54.491173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059a786-ac4e-4b7a-a152-dc91b816072f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:54.497015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c02b2-5b12-4aff-834a-8877b1dbc11a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:54.503415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:54.536321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb2bbd-6f13-4dd7-8228-7b1b4869e212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:54.537070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb2bbd-6f13-4dd7-8228-7b1b4869e212 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:54.537349 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb2bbd-6f13-4dd7-8228-7b1b4869e212 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:54.537477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1238a-2b3f-4fd4-9d62-9dc03ecda308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:54.537666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb2bbd-6f13-4dd7-8228-7b1b4869e212 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:54.537958 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1238a-2b3f-4fd4-9d62-9dc03ecda308 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:54.538234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1238a-2b3f-4fd4-9d62-9dc03ecda308 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:54.538329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb2bbd-6f13-4dd7-8228-7b1b4869e212 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:54.538494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1238a-2b3f-4fd4-9d62-9dc03ecda308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:54.538580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb2bbd-6f13-4dd7-8228-7b1b4869e212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:54.539217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1238a-2b3f-4fd4-9d62-9dc03ecda308 None admin] RBAC: Adding query filter params (role.id=47c08bf673724b40aa00a201adb1411b, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=7cd6d2291f934fbdb25c410c2c3abcb6) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:54.539520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1238a-2b3f-4fd4-9d62-9dc03ecda308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:54.539978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb2bbd-6f13-4dd7-8228-7b1b4869e212 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:54.540950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1238a-2b3f-4fd4-9d62-9dc03ecda308 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:54.556405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 525/1048] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:54 2022] GET /identity/v3/role_assignments?user.id=7cd6d2291f934fbdb25c410c2c3abcb6&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=47c08bf673724b40aa00a201adb1411b => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:54.556625 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 524/1049] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:58:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:54.560289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1238a-2b3f-4fd4-9d62-9dc03ecda308 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:54.651579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:54.677491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1238a-2b3f-4fd4-9d62-9dc03ecda308 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:54.718546 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff39fd2-d20e-44a8-9a19-eab0a0b21a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:54.719180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff39fd2-d20e-44a8-9a19-eab0a0b21a14 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:54.719459 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff39fd2-d20e-44a8-9a19-eab0a0b21a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:54.719730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff39fd2-d20e-44a8-9a19-eab0a0b21a14 None admin] PATH_INFO: `/v3/users/83f0f8bb2afb4235b2e62245e5a84d91` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:54.733862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff39fd2-d20e-44a8-9a19-eab0a0b21a14 None admin] RBAC: Authorizing `identity:get_user(user_id=83f0f8bb2afb4235b2e62245e5a84d91)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:54.735316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff39fd2-d20e-44a8-9a19-eab0a0b21a14 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:54.737894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 526/1050] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:58:54 2022] GET /identity/v3/users/83f0f8bb2afb4235b2e62245e5a84d91 => generated 285 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:54.743518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb2bbd-6f13-4dd7-8228-7b1b4869e212 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:54.845534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb2bbd-6f13-4dd7-8228-7b1b4869e212 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:54.881177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2a85d-f463-463e-abb6-ece6f70f4410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:54.881787 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2a85d-f463-463e-abb6-ece6f70f4410 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:54.882068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2a85d-f463-463e-abb6-ece6f70f4410 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:54.882333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2a85d-f463-463e-abb6-ece6f70f4410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:54.927234 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:54.953514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2a85d-f463-463e-abb6-ece6f70f4410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:54.955336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2a85d-f463-463e-abb6-ece6f70f4410 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:55.063435 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 525/1051] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:54 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:58:55.065282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff39fd2-d20e-44a8-9a19-eab0a0b21a14 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:55.165658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff39fd2-d20e-44a8-9a19-eab0a0b21a14 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:55.200751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08450c-07cd-4170-87bd-a66208e3eb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:55.201319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08450c-07cd-4170-87bd-a66208e3eb34 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:55.201597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08450c-07cd-4170-87bd-a66208e3eb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:55.201865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08450c-07cd-4170-87bd-a66208e3eb34 None admin] PATH_INFO: `/v3/projects/dab7430a5ecc4959b3b8e9cd63212420` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:55.202947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08450c-07cd-4170-87bd-a66208e3eb34 None admin] RBAC: Authorizing `identity:get_project(project_id=dab7430a5ecc4959b3b8e9cd63212420)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:55.205026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08450c-07cd-4170-87bd-a66208e3eb34 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:55.211801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 527/1052] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:58:55 2022] GET /identity/v3/projects/dab7430a5ecc4959b3b8e9cd63212420 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:55.212769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2a85d-f463-463e-abb6-ece6f70f4410 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:55.289893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:55.323542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2a85d-f463-463e-abb6-ece6f70f4410 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:55.359505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90585b-231b-49db-ab22-fee87e084dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:55.360107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90585b-231b-49db-ab22-fee87e084dcd None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:55.360427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90585b-231b-49db-ab22-fee87e084dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:55.360784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90585b-231b-49db-ab22-fee87e084dcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:55.361848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90585b-231b-49db-ab22-fee87e084dcd None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=dab7430a5ecc4959b3b8e9cd63212420, user.id=83f0f8bb2afb4235b2e62245e5a84d91) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:55.362158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90585b-231b-49db-ab22-fee87e084dcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:55.363640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90585b-231b-49db-ab22-fee87e084dcd None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:55.376951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 526/1053] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:58:55 2022] GET /identity/v3/role_assignments?user.id=83f0f8bb2afb4235b2e62245e5a84d91&scope.project.id=dab7430a5ecc4959b3b8e9cd63212420&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:55.494821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc050b24-0ad4-4dbe-bc2d-47ad30ffd7e1 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:55.495018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc050b24-0ad4-4dbe-bc2d-47ad30ffd7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:55.495291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc050b24-0ad4-4dbe-bc2d-47ad30ffd7e1 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:55.496900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 528/1054] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:55.503167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e860299-1b7d-477b-8742-80cba15cdd3d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:55.503355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e860299-1b7d-477b-8742-80cba15cdd3d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:55.503628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e860299-1b7d-477b-8742-80cba15cdd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:55.522630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3e860299-1b7d-477b-8742-80cba15cdd3d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:55.523860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3e860299-1b7d-477b-8742-80cba15cdd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:55.596718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 527/1055] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:55.873563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc050b24-0ad4-4dbe-bc2d-47ad30ffd7e1 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:55.877242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:55.979580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc050b24-0ad4-4dbe-bc2d-47ad30ffd7e1 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:56.014217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ff593-3ac2-447b-95f3-14b96ee0f7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:56.014848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ff593-3ac2-447b-95f3-14b96ee0f7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:56.015130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ff593-3ac2-447b-95f3-14b96ee0f7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:56.015403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ff593-3ac2-447b-95f3-14b96ee0f7a8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:56.015987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ff593-3ac2-447b-95f3-14b96ee0f7a8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:56.018356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ff593-3ac2-447b-95f3-14b96ee0f7a8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-1b5ff593-3ac2-447b-95f3-14b96ee0f7a8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:56.027746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:56.029010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:58:56.030063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:56.033842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 529/1056] 10.222.0.30 () {56 vars in 1105 bytes} [Wed Aug 24 09:58:55 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:56.037965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e860299-1b7d-477b-8742-80cba15cdd3d None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:56.109650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:56.146448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e860299-1b7d-477b-8742-80cba15cdd3d None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:56.180729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bf82a-f5d0-45df-89e2-8ce9c7fd45c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:56.181354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bf82a-f5d0-45df-89e2-8ce9c7fd45c6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:56.181633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bf82a-f5d0-45df-89e2-8ce9c7fd45c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:56.181901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bf82a-f5d0-45df-89e2-8ce9c7fd45c6 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:56.182515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bf82a-f5d0-45df-89e2-8ce9c7fd45c6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:56.182811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bf82a-f5d0-45df-89e2-8ce9c7fd45c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:56.184229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bf82a-f5d0-45df-89e2-8ce9c7fd45c6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:56.192909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 528/1057] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:56.195530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ff593-3ac2-447b-95f3-14b96ee0f7a8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:56.270873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:56.310480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ff593-3ac2-447b-95f3-14b96ee0f7a8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:56.348852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa840eb-5d9d-4636-98a9-77160256c8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:56.349563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa840eb-5d9d-4636-98a9-77160256c8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:56.349840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa840eb-5d9d-4636-98a9-77160256c8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:56.350116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa840eb-5d9d-4636-98a9-77160256c8e7 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:56.350745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa840eb-5d9d-4636-98a9-77160256c8e7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:56.351043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa840eb-5d9d-4636-98a9-77160256c8e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:56.352406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa840eb-5d9d-4636-98a9-77160256c8e7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:56.357959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 530/1058] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:56.364226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bf82a-f5d0-45df-89e2-8ce9c7fd45c6 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:56.458216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc2c7c-528d-4875-afdd-d7d8a575bfe8 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:56.458424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc2c7c-528d-4875-afdd-d7d8a575bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:56.458687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc2c7c-528d-4875-afdd-d7d8a575bfe8 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:56.460152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 531/1059] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:58:56.465019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562243c-6fb4-48bf-b7cc-b1f632f49923 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:56.465307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562243c-6fb4-48bf-b7cc-b1f632f49923 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:56.465582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562243c-6fb4-48bf-b7cc-b1f632f49923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:56.471191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bf82a-f5d0-45df-89e2-8ce9c7fd45c6 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:56.486075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-2562243c-6fb4-48bf-b7cc-b1f632f49923 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:56.487293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2562243c-6fb4-48bf-b7cc-b1f632f49923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:56.507232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:56.513056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a588e-ae86-4bcb-8f64-f7525cf2fe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:56.513664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a588e-ae86-4bcb-8f64-f7525cf2fe4a None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:56.513935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a588e-ae86-4bcb-8f64-f7525cf2fe4a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:56.514202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a588e-ae86-4bcb-8f64-f7525cf2fe4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:56.515080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a588e-ae86-4bcb-8f64-f7525cf2fe4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:56.516216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a588e-ae86-4bcb-8f64-f7525cf2fe4a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:56.560307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 532/1060] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:58:56.602667 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 529/1061] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 09:58:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:58:56.837656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562243c-6fb4-48bf-b7cc-b1f632f49923 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:56.899237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:56.942530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562243c-6fb4-48bf-b7cc-b1f632f49923 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:56.983773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c1a80-3822-43cf-9b34-c0a3e11eeb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:56.984402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c1a80-3822-43cf-9b34-c0a3e11eeb63 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:56.984702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c1a80-3822-43cf-9b34-c0a3e11eeb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:56.985009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c1a80-3822-43cf-9b34-c0a3e11eeb63 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:56.995265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c1a80-3822-43cf-9b34-c0a3e11eeb63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-991c1a80-3822-43cf-9b34-c0a3e11eeb63 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:56.999911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:57.000830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:58:57.002210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:57.004946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 533/1062] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Aug 24 09:58:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:57.007445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a588e-ae86-4bcb-8f64-f7525cf2fe4a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:57.115128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a588e-ae86-4bcb-8f64-f7525cf2fe4a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:57.150105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365bba9-4602-41cd-a11b-c61c5bfda5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:57.150713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365bba9-4602-41cd-a11b-c61c5bfda5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:57.150991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365bba9-4602-41cd-a11b-c61c5bfda5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:57.151262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365bba9-4602-41cd-a11b-c61c5bfda5f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:57.151920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365bba9-4602-41cd-a11b-c61c5bfda5f1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:57.152218 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365bba9-4602-41cd-a11b-c61c5bfda5f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:57.153592 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365bba9-4602-41cd-a11b-c61c5bfda5f1 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:57.166950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 530/1063] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Aug 24 09:58:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:57.168170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c1a80-3822-43cf-9b34-c0a3e11eeb63 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:57.276690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c1a80-3822-43cf-9b34-c0a3e11eeb63 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:57.297273 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:57.315805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704faab3-0f32-4f14-a4cf-616ac2e5b4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:57.316436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704faab3-0f32-4f14-a4cf-616ac2e5b4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:57.316720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704faab3-0f32-4f14-a4cf-616ac2e5b4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:57.317059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704faab3-0f32-4f14-a4cf-616ac2e5b4a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:57.326851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704faab3-0f32-4f14-a4cf-616ac2e5b4a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:57.328332 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704faab3-0f32-4f14-a4cf-616ac2e5b4a2 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:57.336918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f5402-a20c-4415-8ce2-e6affb9a61d8 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:57.337238 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f5402-a20c-4415-8ce2-e6affb9a61d8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:57.337521 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f5402-a20c-4415-8ce2-e6affb9a61d8 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:57.339096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 531/1064] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:57.344032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748b64-b3e0-45d3-9edb-bfd0d95abaa8 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:57.344325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748b64-b3e0-45d3-9edb-bfd0d95abaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-704faab3-0f32-4f14-a4cf-616ac2e5b4a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:57.344575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:57.349390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:57.350292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:57.350292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:57.350292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:57.350475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748b64-b3e0-45d3-9edb-bfd0d95abaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:57.350546 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 534/1065] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:57.352668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704faab3-0f32-4f14-a4cf-616ac2e5b4a2 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:57.366738 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-41748b64-b3e0-45d3-9edb-bfd0d95abaa8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:57.367877 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-41748b64-b3e0-45d3-9edb-bfd0d95abaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:57.413400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:57.436897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 532/1066] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:57.466201 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704faab3-0f32-4f14-a4cf-616ac2e5b4a2 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:57.502335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c50da3-a942-4481-826c-f73cb79a7c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:57.503202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50da3-a942-4481-826c-f73cb79a7c41 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:57.503638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50da3-a942-4481-826c-f73cb79a7c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:57.504056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50da3-a942-4481-826c-f73cb79a7c41 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:57.504845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c50da3-a942-4481-826c-f73cb79a7c41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:57.505302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c50da3-a942-4481-826c-f73cb79a7c41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:57.506698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c50da3-a942-4481-826c-f73cb79a7c41 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:57.519897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 535/1067] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:57.525398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41748b64-b3e0-45d3-9edb-bfd0d95abaa8 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:57.631740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41748b64-b3e0-45d3-9edb-bfd0d95abaa8 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:57.667846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc847f7-0739-460a-aaca-49b88917ba48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:57.668473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc847f7-0739-460a-aaca-49b88917ba48 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:57.668749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc847f7-0739-460a-aaca-49b88917ba48 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:57.669052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc847f7-0739-460a-aaca-49b88917ba48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:57.676681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc847f7-0739-460a-aaca-49b88917ba48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:57.678439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc847f7-0739-460a-aaca-49b88917ba48 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-ebc847f7-0739-460a-aaca-49b88917ba48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:58:57.690829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:57.691818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:57.692765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:57.692765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:57.692765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:58:57.692765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:58:57.692765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:58:57.692765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:58:57.692765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:58:57.692765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:58:57.692765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:58:57.692765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:57.696699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 533/1068] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Aug 24 09:58:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:57.698083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c50da3-a942-4481-826c-f73cb79a7c41 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:57.717013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc847f7-0739-460a-aaca-49b88917ba48 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:57.732403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:57.761440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:57.812166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c50da3-a942-4481-826c-f73cb79a7c41 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:57.834599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc847f7-0739-460a-aaca-49b88917ba48 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:57.848237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333f76d-fc45-47b7-bf48-a179d44a179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:57.849019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333f76d-fc45-47b7-bf48-a179d44a179b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:57.849304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333f76d-fc45-47b7-bf48-a179d44a179b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:57.849583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333f76d-fc45-47b7-bf48-a179d44a179b None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:57.850298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333f76d-fc45-47b7-bf48-a179d44a179b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:57.850606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333f76d-fc45-47b7-bf48-a179d44a179b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:57.852088 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333f76d-fc45-47b7-bf48-a179d44a179b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:57.871550 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 536/1069] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Aug 24 09:58:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:57.878013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333f76d-fc45-47b7-bf48-a179d44a179b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:57.878599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d5400-97d8-42dd-b6b8-74fb32d49e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:57.879232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d5400-97d8-42dd-b6b8-74fb32d49e57 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:57.879511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d5400-97d8-42dd-b6b8-74fb32d49e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:57.879779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d5400-97d8-42dd-b6b8-74fb32d49e57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:57.891416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d5400-97d8-42dd-b6b8-74fb32d49e57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-295d5400-97d8-42dd-b6b8-74fb32d49e57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:58:57.896844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:57.897798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:57.898673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 534/1070] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:58:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:57.904444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d5400-97d8-42dd-b6b8-74fb32d49e57 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:57.975868 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:57.992763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333f76d-fc45-47b7-bf48-a179d44a179b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:58.007082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:58.011087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d5400-97d8-42dd-b6b8-74fb32d49e57 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:58.029833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a657f9-d9be-4dac-a196-ffd2959d07d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:58.030455 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a657f9-d9be-4dac-a196-ffd2959d07d7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:58.030729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a657f9-d9be-4dac-a196-ffd2959d07d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:58.030996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a657f9-d9be-4dac-a196-ffd2959d07d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:58.042027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a657f9-d9be-4dac-a196-ffd2959d07d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:58.043465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a657f9-d9be-4dac-a196-ffd2959d07d7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:58.049564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15961561-bbfd-4152-a9d0-e5e8746b9856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:58.050177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15961561-bbfd-4152-a9d0-e5e8746b9856 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:58.050450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15961561-bbfd-4152-a9d0-e5e8746b9856 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:58.050715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15961561-bbfd-4152-a9d0-e5e8746b9856 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:58.051584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15961561-bbfd-4152-a9d0-e5e8746b9856 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:58.051708 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15961561-bbfd-4152-a9d0-e5e8746b9856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:58.053018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15961561-bbfd-4152-a9d0-e5e8746b9856 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-b7a657f9-d9be-4dac-a196-ffd2959d07d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:58.060602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:58.067336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:58.068275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:58.068275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:58.068275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:58:58.068474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 535/1072] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:58:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:58.068572 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 537/1072] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:58.069865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15961561-bbfd-4152-a9d0-e5e8746b9856 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:58.070414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a657f9-d9be-4dac-a196-ffd2959d07d7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:58.180567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a657f9-d9be-4dac-a196-ffd2959d07d7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:58.183230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15961561-bbfd-4152-a9d0-e5e8746b9856 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:58.215762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40372f5c-54a3-4f7b-82c6-cc5778169672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:58.216644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40372f5c-54a3-4f7b-82c6-cc5778169672 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:58.216951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40372f5c-54a3-4f7b-82c6-cc5778169672 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:58.217222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40372f5c-54a3-4f7b-82c6-cc5778169672 None admin] PATH_INFO: `/v3/users/83f0f8bb2afb4235b2e62245e5a84d91` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:58.218353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40372f5c-54a3-4f7b-82c6-cc5778169672 None admin] RBAC: Authorizing `identity:get_user(user_id=83f0f8bb2afb4235b2e62245e5a84d91)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:58.218981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507524e-0141-4afc-a4ec-7dd9a02764ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:58.220116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507524e-0141-4afc-a4ec-7dd9a02764ef None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:58.220308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40372f5c-54a3-4f7b-82c6-cc5778169672 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:58.220508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507524e-0141-4afc-a4ec-7dd9a02764ef None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:58.221007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507524e-0141-4afc-a4ec-7dd9a02764ef None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:58.221718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507524e-0141-4afc-a4ec-7dd9a02764ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:58.226421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507524e-0141-4afc-a4ec-7dd9a02764ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:58.226554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 538/1073] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:58:58 2022] GET /identity/v3/users/83f0f8bb2afb4235b2e62245e5a84d91 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:58.226646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507524e-0141-4afc-a4ec-7dd9a02764ef None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:58.227421 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40372f5c-54a3-4f7b-82c6-cc5778169672 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:58.237912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:58.243155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 536/1074] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:58:58.244101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507524e-0141-4afc-a4ec-7dd9a02764ef None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:58.269955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:58.346830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40372f5c-54a3-4f7b-82c6-cc5778169672 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:58.353412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507524e-0141-4afc-a4ec-7dd9a02764ef None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:58.382271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571be81-db95-41bb-be48-5f5940758d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:58.382904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571be81-db95-41bb-be48-5f5940758d3a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:58.383178 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571be81-db95-41bb-be48-5f5940758d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:58.383447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571be81-db95-41bb-be48-5f5940758d3a None admin] PATH_INFO: `/v3/projects/dab7430a5ecc4959b3b8e9cd63212420` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:58.384532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571be81-db95-41bb-be48-5f5940758d3a None admin] RBAC: Authorizing `identity:get_project(project_id=dab7430a5ecc4959b3b8e9cd63212420)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:58.386000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571be81-db95-41bb-be48-5f5940758d3a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:58.392589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 539/1075] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:58:58 2022] GET /identity/v3/projects/dab7430a5ecc4959b3b8e9cd63212420 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:58.393680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571be81-db95-41bb-be48-5f5940758d3a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:58.395474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577750f8-9b2e-4e4b-999a-ba6a5d52ee1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:58.396118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577750f8-9b2e-4e4b-999a-ba6a5d52ee1d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:58.396419 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577750f8-9b2e-4e4b-999a-ba6a5d52ee1d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:58.396699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577750f8-9b2e-4e4b-999a-ba6a5d52ee1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:58.404353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577750f8-9b2e-4e4b-999a-ba6a5d52ee1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:58.406241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577750f8-9b2e-4e4b-999a-ba6a5d52ee1d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-577750f8-9b2e-4e4b-999a-ba6a5d52ee1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:58:58.418854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:58.419807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:58.420751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:58:58.420751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:58:58.420751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:58:58.420751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:58:58.420751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:58:58.420751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:58:58.420751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:58:58.420751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:58.420751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 537/1076] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:58:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:58:58.426557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577750f8-9b2e-4e4b-999a-ba6a5d52ee1d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:58.478241 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:58.508195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571be81-db95-41bb-be48-5f5940758d3a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:58.514096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:58.534609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577750f8-9b2e-4e4b-999a-ba6a5d52ee1d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:58.546421 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa836e-1148-4be2-8d23-95f710697bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:58.547085 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa836e-1148-4be2-8d23-95f710697bdd None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:58.547521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa836e-1148-4be2-8d23-95f710697bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:58.548367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa836e-1148-4be2-8d23-95f710697bdd None admin] PATH_INFO: `/v3/projects/dab7430a5ecc4959b3b8e9cd63212420/users/83f0f8bb2afb4235b2e62245e5a84d91/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:58.550826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa836e-1148-4be2-8d23-95f710697bdd None admin] RBAC: Authorizing `identity:create_grant(project_id=dab7430a5ecc4959b3b8e9cd63212420, user_id=83f0f8bb2afb4235b2e62245e5a84d91, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:58.552257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa836e-1148-4be2-8d23-95f710697bdd None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:58.569686 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997d5da-2eac-4359-ac90-bb2966981b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:58.571013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997d5da-2eac-4359-ac90-bb2966981b24 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:58.571013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997d5da-2eac-4359-ac90-bb2966981b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:58.571013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997d5da-2eac-4359-ac90-bb2966981b24 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:58.571483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997d5da-2eac-4359-ac90-bb2966981b24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:58.571783 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997d5da-2eac-4359-ac90-bb2966981b24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:58.573222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997d5da-2eac-4359-ac90-bb2966981b24 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:58.590021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 538/1077] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:58:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:58:58.590021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537670c-a208-4281-bc87-000801411933 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:58.590021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537670c-a208-4281-bc87-000801411933 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:58.590021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537670c-a208-4281-bc87-000801411933 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:58.590021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 539/1078] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:58:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:58.590939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537670c-a208-4281-bc87-000801411933 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:58.630638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 540/1079] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:58:58 2022] PUT /identity/v3/projects/dab7430a5ecc4959b3b8e9cd63212420/users/83f0f8bb2afb4235b2e62245e5a84d91/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:58:58.631894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124c9cf-1663-430c-a7be-199852579f64 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:58.632185 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124c9cf-1663-430c-a7be-199852579f64 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:58.632464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124c9cf-1663-430c-a7be-199852579f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:58.652064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-0124c9cf-1663-430c-a7be-199852579f64 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:58.653642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0124c9cf-1663-430c-a7be-199852579f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:58.711884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537670c-a208-4281-bc87-000801411933 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:58.740639 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 541/1080] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:58:58.756657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040bf31-650e-4bd7-831b-2e3538ce5497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:58.757509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040bf31-650e-4bd7-831b-2e3538ce5497 None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:58.757856 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040bf31-650e-4bd7-831b-2e3538ce5497 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:58.758201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040bf31-650e-4bd7-831b-2e3538ce5497 None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/7cd6d2291f934fbdb25c410c2c3abcb6/roles/47c08bf673724b40aa00a201adb1411b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:58.760451 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040bf31-650e-4bd7-831b-2e3538ce5497 None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=7cd6d2291f934fbdb25c410c2c3abcb6, role_id=47c08bf673724b40aa00a201adb1411b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:58.762024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040bf31-650e-4bd7-831b-2e3538ce5497 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:58.861138 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:58.866182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 540/1081] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:58:58 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/7cd6d2291f934fbdb25c410c2c3abcb6/roles/47c08bf673724b40aa00a201adb1411b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:58:59.018659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124c9cf-1663-430c-a7be-199852579f64 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:59.093353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:59.131318 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124c9cf-1663-430c-a7be-199852579f64 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:59.166265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b7c6f-4ad0-46b4-8636-518b6a49e2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:59.166913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b7c6f-4ad0-46b4-8636-518b6a49e2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:59.167192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b7c6f-4ad0-46b4-8636-518b6a49e2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:59.167464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b7c6f-4ad0-46b4-8636-518b6a49e2a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:59.236611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b7c6f-4ad0-46b4-8636-518b6a49e2a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:59.238080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b7c6f-4ad0-46b4-8636-518b6a49e2a1 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:59.309759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:59.352925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 542/1082] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:58:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:58:59.355662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040bf31-650e-4bd7-831b-2e3538ce5497 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:59.460191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040bf31-650e-4bd7-831b-2e3538ce5497 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:59.495355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643f1d3-30ab-4d65-ad36-906d4a618b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:59.495996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643f1d3-30ab-4d65-ad36-906d4a618b1e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:59.496278 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643f1d3-30ab-4d65-ad36-906d4a618b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:59.496553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643f1d3-30ab-4d65-ad36-906d4a618b1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:59.507027 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643f1d3-30ab-4d65-ad36-906d4a618b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:59.508472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643f1d3-30ab-4d65-ad36-906d4a618b1e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-a643f1d3-30ab-4d65-ad36-906d4a618b1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:58:59.523694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:58:59.524690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:58:59.525660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:58:59.525660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:58:59.525660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:58:59.526530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 541/1083] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:58:59.530820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b7c6f-4ad0-46b4-8636-518b6a49e2a1 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:59.633495 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b7c6f-4ad0-46b4-8636-518b6a49e2a1 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:59.667652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3e8d9-042e-49c8-845e-661bfddfa5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:59.668254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3e8d9-042e-49c8-845e-661bfddfa5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:59.668621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3e8d9-042e-49c8-845e-661bfddfa5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:59.668918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3e8d9-042e-49c8-845e-661bfddfa5b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:59.669581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3e8d9-042e-49c8-845e-661bfddfa5b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:58:59.669851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3e8d9-042e-49c8-845e-661bfddfa5b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:59.671203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3e8d9-042e-49c8-845e-661bfddfa5b0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:59.688903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 543/1084] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:00.693362 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10138fb-0ebd-4703-aacd-1160d8399963 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:00.693922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10138fb-0ebd-4703-aacd-1160d8399963 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:00.693922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10138fb-0ebd-4703-aacd-1160d8399963 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:00.695252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 542/1085] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:00.700733 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5bc77-75b3-4f17-9f38-8023772d70ed None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:00.701208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5bc77-75b3-4f17-9f38-8023772d70ed None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:00.701337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5bc77-75b3-4f17-9f38-8023772d70ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:00.719905 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a3d5bc77-75b3-4f17-9f38-8023772d70ed None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:00.721021 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a3d5bc77-75b3-4f17-9f38-8023772d70ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:00.738820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:00.789356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 544/1086] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:00.839757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bab31-d121-46c1-82ed-3c34b8ad238a None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:00.840151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bab31-d121-46c1-82ed-3c34b8ad238a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:00.840503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bab31-d121-46c1-82ed-3c34b8ad238a None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:00.841959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 543/1087] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:00.848052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb6784-cd11-4fd2-940e-7c98ad8ab821 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:00.848339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb6784-cd11-4fd2-940e-7c98ad8ab821 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:00.848613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb6784-cd11-4fd2-940e-7c98ad8ab821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:00.866697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1fdb6784-cd11-4fd2-940e-7c98ad8ab821 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:00.867871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1fdb6784-cd11-4fd2-940e-7c98ad8ab821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:00.951775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 545/1088] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:01.069133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1bab31-d121-46c1-82ed-3c34b8ad238a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:01.124621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:01.173791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1bab31-d121-46c1-82ed-3c34b8ad238a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:01.209442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b5ae5-449c-4fea-a49b-ab76d0e7c90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:01.210415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b5ae5-449c-4fea-a49b-ab76d0e7c90e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:01.210841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b5ae5-449c-4fea-a49b-ab76d0e7c90e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:01.211119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b5ae5-449c-4fea-a49b-ab76d0e7c90e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:01.221959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb6784-cd11-4fd2-940e-7c98ad8ab821 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:01.222726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b5ae5-449c-4fea-a49b-ab76d0e7c90e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-143b5ae5-449c-4fea-a49b-ab76d0e7c90e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:59:01.227429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:01.228543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:59:01.229471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 544/1089] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:59:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:59:01.234484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b5ae5-449c-4fea-a49b-ab76d0e7c90e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:01.322625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:01.338401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb6784-cd11-4fd2-940e-7c98ad8ab821 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:01.360776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b5ae5-449c-4fea-a49b-ab76d0e7c90e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:01.374326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:01.381053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95ffc3-b2ef-4472-a811-c9e0f2ecda9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:01.381699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95ffc3-b2ef-4472-a811-c9e0f2ecda9b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:01.382001 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95ffc3-b2ef-4472-a811-c9e0f2ecda9b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:01.382275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95ffc3-b2ef-4472-a811-c9e0f2ecda9b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:01.392705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95ffc3-b2ef-4472-a811-c9e0f2ecda9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:59:01.396439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898fd08-74fd-4f99-a61e-df48d675a978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:01.397064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898fd08-74fd-4f99-a61e-df48d675a978 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:01.397342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898fd08-74fd-4f99-a61e-df48d675a978 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-0f95ffc3-b2ef-4472-a811-c9e0f2ecda9b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:59:01.397452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:01.398483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:59:01.399389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:59:01.400039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898fd08-74fd-4f99-a61e-df48d675a978 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:01.400039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898fd08-74fd-4f99-a61e-df48d675a978 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:01.400165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 546/1090] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Aug 24 09:59:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:59:01.400233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898fd08-74fd-4f99-a61e-df48d675a978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:01.400233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898fd08-74fd-4f99-a61e-df48d675a978 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:01.404817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95ffc3-b2ef-4472-a811-c9e0f2ecda9b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:01.410494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 545/1091] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:59:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:01.416501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898fd08-74fd-4f99-a61e-df48d675a978 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:01.512414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95ffc3-b2ef-4472-a811-c9e0f2ecda9b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:01.523399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898fd08-74fd-4f99-a61e-df48d675a978 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:01.554423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c89ba-129f-472b-bdee-07ea34c3c707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:01.555033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c89ba-129f-472b-bdee-07ea34c3c707 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:01.555414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c89ba-129f-472b-bdee-07ea34c3c707 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:01.555908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c89ba-129f-472b-bdee-07ea34c3c707 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:01.557045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c89ba-129f-472b-bdee-07ea34c3c707 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:01.557557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c89ba-129f-472b-bdee-07ea34c3c707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:01.559937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c89ba-129f-472b-bdee-07ea34c3c707 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:01.560206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c23722-c202-45d1-9fef-369deafc7602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:01.560832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c23722-c202-45d1-9fef-369deafc7602 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:01.561138 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c23722-c202-45d1-9fef-369deafc7602 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:01.561408 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c23722-c202-45d1-9fef-369deafc7602 None admin] PATH_INFO: `/v3/users/83f0f8bb2afb4235b2e62245e5a84d91` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:01.562651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c23722-c202-45d1-9fef-369deafc7602 None admin] RBAC: Authorizing `identity:get_user(user_id=83f0f8bb2afb4235b2e62245e5a84d91)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:01.565587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c23722-c202-45d1-9fef-369deafc7602 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:01.567371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 546/1092] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:01 2022] GET /identity/v3/users/83f0f8bb2afb4235b2e62245e5a84d91 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:59:01.577253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 547/1093] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Aug 24 09:59:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:59:01.577463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c23722-c202-45d1-9fef-369deafc7602 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:01.578367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c89ba-129f-472b-bdee-07ea34c3c707 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:01.587919 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:01.624450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:01.689197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c23722-c202-45d1-9fef-369deafc7602 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:01.689819 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c89ba-129f-472b-bdee-07ea34c3c707 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:01.726423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b01a1-d648-4b32-af85-3d58ae1f864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:01.727103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b01a1-d648-4b32-af85-3d58ae1f864a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:01.727325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b01a1-d648-4b32-af85-3d58ae1f864a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:01.727600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b01a1-d648-4b32-af85-3d58ae1f864a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:01.729754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f233aeb-86ad-49f9-80f8-eb1268015b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:01.730372 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f233aeb-86ad-49f9-80f8-eb1268015b7b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:01.730651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f233aeb-86ad-49f9-80f8-eb1268015b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:01.730926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f233aeb-86ad-49f9-80f8-eb1268015b7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:01.738942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b01a1-d648-4b32-af85-3d58ae1f864a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:01.740401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b01a1-d648-4b32-af85-3d58ae1f864a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-2d6b01a1-d648-4b32-af85-3d58ae1f864a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:59:01.755504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:01.757124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:59:01.758477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:59:01.758477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:59:01.758477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:59:01.758477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 548/1094] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:59:01.762361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b01a1-d648-4b32-af85-3d58ae1f864a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:01.824056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f233aeb-86ad-49f9-80f8-eb1268015b7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:01.824056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f233aeb-86ad-49f9-80f8-eb1268015b7b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:01.826208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:01.865182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:01.884632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b01a1-d648-4b32-af85-3d58ae1f864a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:01.920883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f7194-282e-4815-a32e-0ca9b6152c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:01.921498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f7194-282e-4815-a32e-0ca9b6152c64 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:01.921785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f7194-282e-4815-a32e-0ca9b6152c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:01.922061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f7194-282e-4815-a32e-0ca9b6152c64 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:01.922877 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f7194-282e-4815-a32e-0ca9b6152c64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:01.923196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f7194-282e-4815-a32e-0ca9b6152c64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:01.924617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f7194-282e-4815-a32e-0ca9b6152c64 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:01.931934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 547/1095] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:59:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:59:01.942754 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 549/1096] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:59:01.943111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f233aeb-86ad-49f9-80f8-eb1268015b7b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:01.943760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f7194-282e-4815-a32e-0ca9b6152c64 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:02.052846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f233aeb-86ad-49f9-80f8-eb1268015b7b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:02.066131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f7194-282e-4815-a32e-0ca9b6152c64 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:02.085837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:02.091929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e8aee-cacf-4e3e-8e46-3dcc00797ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:02.093226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e8aee-cacf-4e3e-8e46-3dcc00797ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:02.093226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e8aee-cacf-4e3e-8e46-3dcc00797ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:02.093226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e8aee-cacf-4e3e-8e46-3dcc00797ac5 None admin] PATH_INFO: `/v3/projects/dab7430a5ecc4959b3b8e9cd63212420` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:02.094580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e8aee-cacf-4e3e-8e46-3dcc00797ac5 None admin] RBAC: Authorizing `identity:get_project(project_id=dab7430a5ecc4959b3b8e9cd63212420)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:02.096205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e8aee-cacf-4e3e-8e46-3dcc00797ac5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:02.098027 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 548/1097] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:59:01 2022] GET /identity/v3/projects/dab7430a5ecc4959b3b8e9cd63212420 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:02.104249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e8aee-cacf-4e3e-8e46-3dcc00797ac5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:02.105605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a3fe4-f682-4035-9eb8-763717a49d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:02.106447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a3fe4-f682-4035-9eb8-763717a49d3b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:02.107129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a3fe4-f682-4035-9eb8-763717a49d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:02.107310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a3fe4-f682-4035-9eb8-763717a49d3b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:02.116098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a3fe4-f682-4035-9eb8-763717a49d3b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:02.117579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a3fe4-f682-4035-9eb8-763717a49d3b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:02.118719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-869a3fe4-f682-4035-9eb8-763717a49d3b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:59:02.131213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:02.132410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 550/1098] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Aug 24 09:59:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:59:02.142813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a3fe4-f682-4035-9eb8-763717a49d3b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:02.225355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e8aee-cacf-4e3e-8e46-3dcc00797ac5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:02.258888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a3fe4-f682-4035-9eb8-763717a49d3b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:02.272205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd99c5e-00ba-4dac-90e5-970dc8c11b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:02.273105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd99c5e-00ba-4dac-90e5-970dc8c11b4e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:02.273643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd99c5e-00ba-4dac-90e5-970dc8c11b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:02.274024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd99c5e-00ba-4dac-90e5-970dc8c11b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:02.274944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd99c5e-00ba-4dac-90e5-970dc8c11b4e None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=dab7430a5ecc4959b3b8e9cd63212420, user.id=83f0f8bb2afb4235b2e62245e5a84d91) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:02.275383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd99c5e-00ba-4dac-90e5-970dc8c11b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:02.277606 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd99c5e-00ba-4dac-90e5-970dc8c11b4e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:02.300901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 549/1099] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:59:02 2022] GET /identity/v3/role_assignments?user.id=83f0f8bb2afb4235b2e62245e5a84d91&scope.project.id=dab7430a5ecc4959b3b8e9cd63212420&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:59:02.301631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa3bca-a59f-4bbd-86c2-d507ab28ce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:02.301631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa3bca-a59f-4bbd-86c2-d507ab28ce8d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:02.301631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa3bca-a59f-4bbd-86c2-d507ab28ce8d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:02.301631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa3bca-a59f-4bbd-86c2-d507ab28ce8d None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:02.301993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa3bca-a59f-4bbd-86c2-d507ab28ce8d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:02.301993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa3bca-a59f-4bbd-86c2-d507ab28ce8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:02.301993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa3bca-a59f-4bbd-86c2-d507ab28ce8d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:02.322298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 551/1100] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Aug 24 09:59:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:59:02.328933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd99c5e-00ba-4dac-90e5-970dc8c11b4e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:02.427515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:02.453145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd99c5e-00ba-4dac-90e5-970dc8c11b4e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:02.490708 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee6edd-96cb-4b34-977e-f931f885ade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:02.491329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee6edd-96cb-4b34-977e-f931f885ade6 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:02.491600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee6edd-96cb-4b34-977e-f931f885ade6 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:02.491876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee6edd-96cb-4b34-977e-f931f885ade6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:02.564680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee6edd-96cb-4b34-977e-f931f885ade6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:02.566838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee6edd-96cb-4b34-977e-f931f885ade6 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:02.661817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:02.695158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 550/1101] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:59:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:59:02.696721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa3bca-a59f-4bbd-86c2-d507ab28ce8d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:02.731910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:02.799870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa3bca-a59f-4bbd-86c2-d507ab28ce8d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:02.833649 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96588105-5e02-4fc1-9a62-f92a05bae0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:02.834271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96588105-5e02-4fc1-9a62-f92a05bae0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:02.834549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96588105-5e02-4fc1-9a62-f92a05bae0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:02.834818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96588105-5e02-4fc1-9a62-f92a05bae0b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:02.844810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96588105-5e02-4fc1-9a62-f92a05bae0b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:02.846394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96588105-5e02-4fc1-9a62-f92a05bae0b8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-96588105-5e02-4fc1-9a62-f92a05bae0b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:59:02.861593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:02.862573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:59:02.863829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:59:02.863829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:59:02.863829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:59:02.863829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 552/1102] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:59:02.868375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee6edd-96cb-4b34-977e-f931f885ade6 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:02.968076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee6edd-96cb-4b34-977e-f931f885ade6 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:03.002591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdfb3c-0942-44fb-9ef5-245336e3975f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:03.003191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdfb3c-0942-44fb-9ef5-245336e3975f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:03.003467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdfb3c-0942-44fb-9ef5-245336e3975f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:03.003735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdfb3c-0942-44fb-9ef5-245336e3975f None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:03.004380 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdfb3c-0942-44fb-9ef5-245336e3975f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:03.004675 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdfb3c-0942-44fb-9ef5-245336e3975f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:03.006102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdfb3c-0942-44fb-9ef5-245336e3975f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:03.022400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 551/1103] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 09:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:03.023554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96588105-5e02-4fc1-9a62-f92a05bae0b8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:03.111467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:03.123195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96588105-5e02-4fc1-9a62-f92a05bae0b8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:03.157823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761d0a-5d07-4586-ade4-7f766c1e72c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:03.158448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761d0a-5d07-4586-ade4-7f766c1e72c1 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:03.158729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761d0a-5d07-4586-ade4-7f766c1e72c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:03.159258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761d0a-5d07-4586-ade4-7f766c1e72c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:03.166584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a761d0a-5d07-4586-ade4-7f766c1e72c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:03.168014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a761d0a-5d07-4586-ade4-7f766c1e72c1 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-6a761d0a-5d07-4586-ade4-7f766c1e72c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:59:03.179511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:03.180596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:03.181829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:59:03.181829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:59:03.181829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:59:03.181829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:59:03.181829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:59:03.181829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:59:03.181829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:59:03.181829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:59:03.181829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 553/1104] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 09:59:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:59:03.186759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdfb3c-0942-44fb-9ef5-245336e3975f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:03.226960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:03.303485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdfb3c-0942-44fb-9ef5-245336e3975f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:03.337642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d47941-b99c-43db-aeb5-4e656be8e86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:03.338243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d47941-b99c-43db-aeb5-4e656be8e86a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:03.338519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d47941-b99c-43db-aeb5-4e656be8e86a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:03.338785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d47941-b99c-43db-aeb5-4e656be8e86a None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:03.339469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d47941-b99c-43db-aeb5-4e656be8e86a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:03.339768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d47941-b99c-43db-aeb5-4e656be8e86a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:03.341992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d47941-b99c-43db-aeb5-4e656be8e86a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:03.355408 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 552/1105] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 09:59:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:59:03.361872 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761d0a-5d07-4586-ade4-7f766c1e72c1 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:03.519039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761d0a-5d07-4586-ade4-7f766c1e72c1 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:03.568075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60170b-64c7-4c4b-8432-ae547be5f644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:03.568996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60170b-64c7-4c4b-8432-ae547be5f644 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:03.569417 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60170b-64c7-4c4b-8432-ae547be5f644 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:03.569801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60170b-64c7-4c4b-8432-ae547be5f644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:03.570534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60170b-64c7-4c4b-8432-ae547be5f644 None admin] RBAC: Adding query filter params (role.id=47c08bf673724b40aa00a201adb1411b, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=7cd6d2291f934fbdb25c410c2c3abcb6) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:03.570878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60170b-64c7-4c4b-8432-ae547be5f644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:03.572364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60170b-64c7-4c4b-8432-ae547be5f644 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:03.580518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 554/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:59:03 2022] GET /identity/v3/role_assignments?user.id=7cd6d2291f934fbdb25c410c2c3abcb6&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=47c08bf673724b40aa00a201adb1411b => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:59:04.282121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d58e78-047c-4959-b95e-33a08c823431 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:04.282831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d58e78-047c-4959-b95e-33a08c823431 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:04.282955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d58e78-047c-4959-b95e-33a08c823431 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:04.284578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 553/1107] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:04.290914 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2439a-9bb5-4389-a0a5-26329680cc69 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:04.291402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2439a-9bb5-4389-a0a5-26329680cc69 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:04.291517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2439a-9bb5-4389-a0a5-26329680cc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:04.311909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-0fa2439a-9bb5-4389-a0a5-26329680cc69 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:04.314228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0fa2439a-9bb5-4389-a0a5-26329680cc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:04.322673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:04.386637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 555/1108] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:04.666662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d58e78-047c-4959-b95e-33a08c823431 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:04.761960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:04.774833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d58e78-047c-4959-b95e-33a08c823431 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:04.813243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624cb8f-e736-409f-b705-d68ec2a7d68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:04.814045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624cb8f-e736-409f-b705-d68ec2a7d68d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:04.814322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624cb8f-e736-409f-b705-d68ec2a7d68d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:04.814596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624cb8f-e736-409f-b705-d68ec2a7d68d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:04.815706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624cb8f-e736-409f-b705-d68ec2a7d68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:04.817156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624cb8f-e736-409f-b705-d68ec2a7d68d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:04.818715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 554/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:59:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:04.824432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2439a-9bb5-4389-a0a5-26329680cc69 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:04.920157 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2439a-9bb5-4389-a0a5-26329680cc69 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:04.953985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f4c30-a85a-473a-9876-549c5ba01eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:04.954677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f4c30-a85a-473a-9876-549c5ba01eeb None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:04.954954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f4c30-a85a-473a-9876-549c5ba01eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:04.955225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f4c30-a85a-473a-9876-549c5ba01eeb None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:04.955855 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f4c30-a85a-473a-9876-549c5ba01eeb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:04.957302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f4c30-a85a-473a-9876-549c5ba01eeb None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:05.076040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:05.082935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 556/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:59:04 2022] POST /identity/v3/users => generated 286 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:07.071436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99757e67-8096-4cee-8b15-65fe8dba58bf None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:07.071990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99757e67-8096-4cee-8b15-65fe8dba58bf None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:07.071990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99757e67-8096-4cee-8b15-65fe8dba58bf None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:07.073387 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 555/1111] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:07.078706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88f955-1476-42c6-9716-a08f5edd2afa None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:07.079128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88f955-1476-42c6-9716-a08f5edd2afa None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:07.079250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88f955-1476-42c6-9716-a08f5edd2afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:07.097510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-fd88f955-1476-42c6-9716-a08f5edd2afa None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:07.098685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-fd88f955-1476-42c6-9716-a08f5edd2afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:07.163950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 557/1112] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:07.438704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99757e67-8096-4cee-8b15-65fe8dba58bf None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:07.540713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99757e67-8096-4cee-8b15-65fe8dba58bf None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:07.580159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73246002-117c-4439-99ab-48a4f0471353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:07.580781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246002-117c-4439-99ab-48a4f0471353 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:07.581091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246002-117c-4439-99ab-48a4f0471353 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:07.581369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246002-117c-4439-99ab-48a4f0471353 None admin] PATH_INFO: `/v3/roles/276ae07995d14b5d8ceb8e646392c2b1` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:07.582825 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73246002-117c-4439-99ab-48a4f0471353 None admin] RBAC: Authorizing `identity:get_role(role_id=276ae07995d14b5d8ceb8e646392c2b1)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:07.584199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73246002-117c-4439-99ab-48a4f0471353 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:07.585557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 556/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:07 2022] GET /identity/v3/roles/276ae07995d14b5d8ceb8e646392c2b1 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:07.591448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88f955-1476-42c6-9716-a08f5edd2afa None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:07.686922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88f955-1476-42c6-9716-a08f5edd2afa None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:07.714408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:07.721903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a98aa6-cac4-4ea3-b1b6-63ec91703235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:07.722522 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a98aa6-cac4-4ea3-b1b6-63ec91703235 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:07.722798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a98aa6-cac4-4ea3-b1b6-63ec91703235 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:07.723066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a98aa6-cac4-4ea3-b1b6-63ec91703235 None admin] PATH_INFO: `/v3/users/b981dfcffc694ef5a0f8acbcd88df3c6` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:07.735649 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a98aa6-cac4-4ea3-b1b6-63ec91703235 None admin] RBAC: Authorizing `identity:get_user(user_id=b981dfcffc694ef5a0f8acbcd88df3c6)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:07.737157 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a98aa6-cac4-4ea3-b1b6-63ec91703235 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:07.738734 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 558/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:07 2022] GET /identity/v3/users/b981dfcffc694ef5a0f8acbcd88df3c6 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:07.744705 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73246002-117c-4439-99ab-48a4f0471353 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:07.776293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:07.845152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73246002-117c-4439-99ab-48a4f0471353 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:07.880319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eb12d-7082-4af8-8a74-b1ed3bb22aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:07.880938 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eb12d-7082-4af8-8a74-b1ed3bb22aea None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:07.881221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eb12d-7082-4af8-8a74-b1ed3bb22aea None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:07.881492 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eb12d-7082-4af8-8a74-b1ed3bb22aea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:07.948448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572eb12d-7082-4af8-8a74-b1ed3bb22aea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:07.949904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572eb12d-7082-4af8-8a74-b1ed3bb22aea None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:07.978608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:08.046768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 557/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:59:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:59:08.053576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a98aa6-cac4-4ea3-b1b6-63ec91703235 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:08.150127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a98aa6-cac4-4ea3-b1b6-63ec91703235 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:08.183154 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35514620-0e66-4089-8ed9-22f10d8ad8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:08.183767 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35514620-0e66-4089-8ed9-22f10d8ad8bf None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:08.184044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35514620-0e66-4089-8ed9-22f10d8ad8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:08.184314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35514620-0e66-4089-8ed9-22f10d8ad8bf None admin] PATH_INFO: `/v3/projects/dab7430a5ecc4959b3b8e9cd63212420` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:08.185432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35514620-0e66-4089-8ed9-22f10d8ad8bf None admin] RBAC: Authorizing `identity:get_project(project_id=dab7430a5ecc4959b3b8e9cd63212420)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:08.186841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35514620-0e66-4089-8ed9-22f10d8ad8bf None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:08.189033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 559/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:59:08 2022] GET /identity/v3/projects/dab7430a5ecc4959b3b8e9cd63212420 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:08.195508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eb12d-7082-4af8-8a74-b1ed3bb22aea None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:08.293576 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eb12d-7082-4af8-8a74-b1ed3bb22aea None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:08.322382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:08.328028 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbc1e2-04d8-4190-905d-01425d7f6711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:08.328713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbc1e2-04d8-4190-905d-01425d7f6711 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:08.329030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbc1e2-04d8-4190-905d-01425d7f6711 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:08.329301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbc1e2-04d8-4190-905d-01425d7f6711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:08.330022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbc1e2-04d8-4190-905d-01425d7f6711 None admin] RBAC: Adding query filter params (role.id=276ae07995d14b5d8ceb8e646392c2b1, scope.project.id=dab7430a5ecc4959b3b8e9cd63212420, user.id=b981dfcffc694ef5a0f8acbcd88df3c6) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:08.330332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbc1e2-04d8-4190-905d-01425d7f6711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:08.331749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbc1e2-04d8-4190-905d-01425d7f6711 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:08.336940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 558/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:59:08 2022] GET /identity/v3/role_assignments?user.id=b981dfcffc694ef5a0f8acbcd88df3c6&scope.project.id=dab7430a5ecc4959b3b8e9cd63212420&role.id=276ae07995d14b5d8ceb8e646392c2b1 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:10.188436 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179011ce-f855-438b-99ef-e216984c0831 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:10.189033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179011ce-f855-438b-99ef-e216984c0831 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:10.189033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179011ce-f855-438b-99ef-e216984c0831 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:10.190357 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 560/1118] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:10.195440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7715c9-a8a0-487e-af6a-2122091a6241 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:10.195851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7715c9-a8a0-487e-af6a-2122091a6241 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:10.195986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7715c9-a8a0-487e-af6a-2122091a6241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:10.216090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-2a7715c9-a8a0-487e-af6a-2122091a6241 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:10.217324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2a7715c9-a8a0-487e-af6a-2122091a6241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:10.286054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 559/1119] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:10.558305 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179011ce-f855-438b-99ef-e216984c0831 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:10.616130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:10.661242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179011ce-f855-438b-99ef-e216984c0831 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:10.695356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78bb96-4d5d-4594-98f9-f1fb2b0cb216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:10.695974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78bb96-4d5d-4594-98f9-f1fb2b0cb216 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:10.696260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78bb96-4d5d-4594-98f9-f1fb2b0cb216 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:10.696552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78bb96-4d5d-4594-98f9-f1fb2b0cb216 None admin] PATH_INFO: `/v3/roles/276ae07995d14b5d8ceb8e646392c2b1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:10.697964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78bb96-4d5d-4594-98f9-f1fb2b0cb216 None admin] RBAC: Authorizing `identity:get_role(role_id=276ae07995d14b5d8ceb8e646392c2b1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:10.699329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78bb96-4d5d-4594-98f9-f1fb2b0cb216 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:10.700669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 561/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:10 2022] GET /identity/v3/roles/276ae07995d14b5d8ceb8e646392c2b1 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:10.707397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7715c9-a8a0-487e-af6a-2122091a6241 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:10.802728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7715c9-a8a0-487e-af6a-2122091a6241 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:10.826354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:10.837190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98674616-8adf-4810-992b-4d714b9a7040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:10.837814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674616-8adf-4810-992b-4d714b9a7040 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:10.838092 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674616-8adf-4810-992b-4d714b9a7040 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:10.839924 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674616-8adf-4810-992b-4d714b9a7040 None admin] PATH_INFO: `/v3/users/b981dfcffc694ef5a0f8acbcd88df3c6` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:10.839924 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98674616-8adf-4810-992b-4d714b9a7040 None admin] RBAC: Authorizing `identity:get_user(user_id=b981dfcffc694ef5a0f8acbcd88df3c6)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:10.840897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98674616-8adf-4810-992b-4d714b9a7040 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:10.842458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 560/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:10 2022] GET /identity/v3/users/b981dfcffc694ef5a0f8acbcd88df3c6 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:10.848419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78bb96-4d5d-4594-98f9-f1fb2b0cb216 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:10.947012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78bb96-4d5d-4594-98f9-f1fb2b0cb216 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:10.969535 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:10.981244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6867648-1868-4270-9e35-28a4c1b1a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:10.981861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6867648-1868-4270-9e35-28a4c1b1a882 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:10.986006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6867648-1868-4270-9e35-28a4c1b1a882 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:10.986006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6867648-1868-4270-9e35-28a4c1b1a882 None admin] PATH_INFO: `/v3/projects/dab7430a5ecc4959b3b8e9cd63212420` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:10.986006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6867648-1868-4270-9e35-28a4c1b1a882 None admin] RBAC: Authorizing `identity:get_project(project_id=dab7430a5ecc4959b3b8e9cd63212420)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:10.986006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6867648-1868-4270-9e35-28a4c1b1a882 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:10.987325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 562/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:59:10 2022] GET /identity/v3/projects/dab7430a5ecc4959b3b8e9cd63212420 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:10.993386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98674616-8adf-4810-992b-4d714b9a7040 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:11.092517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98674616-8adf-4810-992b-4d714b9a7040 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:11.126078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108240b-cdf6-4e34-b0fe-e68e4eac3606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:11.126723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108240b-cdf6-4e34-b0fe-e68e4eac3606 None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:11.127008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108240b-cdf6-4e34-b0fe-e68e4eac3606 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:11.127287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108240b-cdf6-4e34-b0fe-e68e4eac3606 None admin] PATH_INFO: `/v3/projects/dab7430a5ecc4959b3b8e9cd63212420/users/b981dfcffc694ef5a0f8acbcd88df3c6/roles/276ae07995d14b5d8ceb8e646392c2b1` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:11.129445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108240b-cdf6-4e34-b0fe-e68e4eac3606 None admin] RBAC: Authorizing `identity:create_grant(project_id=dab7430a5ecc4959b3b8e9cd63212420, user_id=b981dfcffc694ef5a0f8acbcd88df3c6, role_id=276ae07995d14b5d8ceb8e646392c2b1)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:11.130895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108240b-cdf6-4e34-b0fe-e68e4eac3606 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:11.217955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 561/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:59:10 2022] PUT /identity/v3/projects/dab7430a5ecc4959b3b8e9cd63212420/users/b981dfcffc694ef5a0f8acbcd88df3c6/roles/276ae07995d14b5d8ceb8e646392c2b1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:13.086932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77ce4f-2d41-471e-aa8e-3973f285d6f8 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:13.087505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77ce4f-2d41-471e-aa8e-3973f285d6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:13.087505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77ce4f-2d41-471e-aa8e-3973f285d6f8 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:13.088805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 563/1124] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:13.094467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323271e-b282-4002-a3d4-8c8aa7940cb1 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:13.094867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323271e-b282-4002-a3d4-8c8aa7940cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:13.095103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323271e-b282-4002-a3d4-8c8aa7940cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:13.114139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-9323271e-b282-4002-a3d4-8c8aa7940cb1 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:13.115306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9323271e-b282-4002-a3d4-8c8aa7940cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:13.160940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:13.179823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 562/1125] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:13.447681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77ce4f-2d41-471e-aa8e-3973f285d6f8 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:13.547584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77ce4f-2d41-471e-aa8e-3973f285d6f8 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:13.581424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1c7f6-5d6d-46c1-a310-3241dcfad964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:13.582039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1c7f6-5d6d-46c1-a310-3241dcfad964 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:13.582318 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1c7f6-5d6d-46c1-a310-3241dcfad964 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:13.582589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1c7f6-5d6d-46c1-a310-3241dcfad964 None admin] PATH_INFO: `/v3/roles/276ae07995d14b5d8ceb8e646392c2b1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:13.583987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1c7f6-5d6d-46c1-a310-3241dcfad964 None admin] RBAC: Authorizing `identity:get_role(role_id=276ae07995d14b5d8ceb8e646392c2b1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:13.585378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1c7f6-5d6d-46c1-a310-3241dcfad964 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:13.586677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 564/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:13 2022] GET /identity/v3/roles/276ae07995d14b5d8ceb8e646392c2b1 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:13.592442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323271e-b282-4002-a3d4-8c8aa7940cb1 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:13.691333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323271e-b282-4002-a3d4-8c8aa7940cb1 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:13.724683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f8419-c9e0-4493-bd50-06c522f71b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:13.725342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f8419-c9e0-4493-bd50-06c522f71b38 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:13.725623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f8419-c9e0-4493-bd50-06c522f71b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:13.725897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f8419-c9e0-4493-bd50-06c522f71b38 None admin] PATH_INFO: `/v3/users/b981dfcffc694ef5a0f8acbcd88df3c6` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:13.727006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f8419-c9e0-4493-bd50-06c522f71b38 None admin] RBAC: Authorizing `identity:get_user(user_id=b981dfcffc694ef5a0f8acbcd88df3c6)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:13.728427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f8419-c9e0-4493-bd50-06c522f71b38 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:13.730033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 563/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:13 2022] GET /identity/v3/users/b981dfcffc694ef5a0f8acbcd88df3c6 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:13.736503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1c7f6-5d6d-46c1-a310-3241dcfad964 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:13.793825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:13.834435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1c7f6-5d6d-46c1-a310-3241dcfad964 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:13.868383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4b394-0502-45f0-8346-6584fa55ba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:13.869009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4b394-0502-45f0-8346-6584fa55ba23 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:13.869285 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4b394-0502-45f0-8346-6584fa55ba23 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:13.869556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4b394-0502-45f0-8346-6584fa55ba23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:13.936950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4b394-0502-45f0-8346-6584fa55ba23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:13.938750 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4b394-0502-45f0-8346-6584fa55ba23 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:13.993062 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:14.034378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 565/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:59:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:59:14.040582 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f8419-c9e0-4493-bd50-06c522f71b38 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:14.089811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:14.137762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f8419-c9e0-4493-bd50-06c522f71b38 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:14.172515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d24c7-5088-4f63-9541-8cb1e221ba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:14.173169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d24c7-5088-4f63-9541-8cb1e221ba4d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:14.173450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d24c7-5088-4f63-9541-8cb1e221ba4d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:14.173725 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d24c7-5088-4f63-9541-8cb1e221ba4d None admin] PATH_INFO: `/v3/projects/dab7430a5ecc4959b3b8e9cd63212420` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:14.174861 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d24c7-5088-4f63-9541-8cb1e221ba4d None admin] RBAC: Authorizing `identity:get_project(project_id=dab7430a5ecc4959b3b8e9cd63212420)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:14.176290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d24c7-5088-4f63-9541-8cb1e221ba4d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:14.177972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 564/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:59:14 2022] GET /identity/v3/projects/dab7430a5ecc4959b3b8e9cd63212420 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:14.184089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4b394-0502-45f0-8346-6584fa55ba23 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:14.282382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4b394-0502-45f0-8346-6584fa55ba23 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:14.315681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db934cab-63c4-41e6-81a5-b518d083d99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:14.316279 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db934cab-63c4-41e6-81a5-b518d083d99b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:14.316559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db934cab-63c4-41e6-81a5-b518d083d99b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:14.316832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db934cab-63c4-41e6-81a5-b518d083d99b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:14.317751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db934cab-63c4-41e6-81a5-b518d083d99b None admin] RBAC: Adding query filter params (role.id=276ae07995d14b5d8ceb8e646392c2b1, scope.project.id=dab7430a5ecc4959b3b8e9cd63212420, user.id=b981dfcffc694ef5a0f8acbcd88df3c6) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:14.318058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db934cab-63c4-41e6-81a5-b518d083d99b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:14.319508 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db934cab-63c4-41e6-81a5-b518d083d99b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:14.325115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 566/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:59:14 2022] GET /identity/v3/role_assignments?user.id=b981dfcffc694ef5a0f8acbcd88df3c6&scope.project.id=dab7430a5ecc4959b3b8e9cd63212420&role.id=276ae07995d14b5d8ceb8e646392c2b1 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:16.255597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c4ac-d2ba-4283-9fcb-02b03ac2cfc5 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:16.256183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c4ac-d2ba-4283-9fcb-02b03ac2cfc5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:16.256183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c4ac-d2ba-4283-9fcb-02b03ac2cfc5 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:16.257501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 565/1131] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:16.262829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b36c4-67b4-4130-923a-8754a871626c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:16.263271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b36c4-67b4-4130-923a-8754a871626c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:16.263386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b36c4-67b4-4130-923a-8754a871626c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:16.281786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c66b36c4-67b4-4130-923a-8754a871626c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:16.282957 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c66b36c4-67b4-4130-923a-8754a871626c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:16.299972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:16.351364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 567/1132] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:16.628979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c4ac-d2ba-4283-9fcb-02b03ac2cfc5 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:16.728419 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c4ac-d2ba-4283-9fcb-02b03ac2cfc5 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:16.745085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:16.761561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b55db-f8b2-495c-a6ab-e51b9d19cd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:16.762181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b55db-f8b2-495c-a6ab-e51b9d19cd06 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:16.762460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b55db-f8b2-495c-a6ab-e51b9d19cd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:16.762733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b55db-f8b2-495c-a6ab-e51b9d19cd06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:16.763831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b55db-f8b2-495c-a6ab-e51b9d19cd06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:16.765417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b55db-f8b2-495c-a6ab-e51b9d19cd06 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:16.767462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 566/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:59:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:16.773378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b36c4-67b4-4130-923a-8754a871626c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:16.870773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b36c4-67b4-4130-923a-8754a871626c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:16.903450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58ab5a-43fe-45da-aa56-34ce62aa16c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:16.903976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58ab5a-43fe-45da-aa56-34ce62aa16c6 None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:16.904253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58ab5a-43fe-45da-aa56-34ce62aa16c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:16.904523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58ab5a-43fe-45da-aa56-34ce62aa16c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:16.905203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58ab5a-43fe-45da-aa56-34ce62aa16c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:16.907172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58ab5a-43fe-45da-aa56-34ce62aa16c6 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:17.024844 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 568/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:59:16 2022] POST /identity/v3/projects => generated 311 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:18.958740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bbe33-2dae-4587-a87b-0454707f17c4 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:18.959275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bbe33-2dae-4587-a87b-0454707f17c4 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:18.959275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bbe33-2dae-4587-a87b-0454707f17c4 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:18.960606 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 567/1135] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:18.965795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c786cd9-4ac9-4c09-8a10-268999c470b5 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:18.966213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c786cd9-4ac9-4c09-8a10-268999c470b5 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:18.966354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c786cd9-4ac9-4c09-8a10-268999c470b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:18.974324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:18.985519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1c786cd9-4ac9-4c09-8a10-268999c470b5 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:18.988445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1c786cd9-4ac9-4c09-8a10-268999c470b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:19.052538 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 569/1136] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:19.319081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bbe33-2dae-4587-a87b-0454707f17c4 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:19.416829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bbe33-2dae-4587-a87b-0454707f17c4 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:19.450651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d29a3-4b82-4f27-a00e-038b9b5185ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:19.451268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d29a3-4b82-4f27-a00e-038b9b5185ba None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:19.451543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d29a3-4b82-4f27-a00e-038b9b5185ba None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:19.451814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d29a3-4b82-4f27-a00e-038b9b5185ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:19.452985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d29a3-4b82-4f27-a00e-038b9b5185ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:19.454411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d29a3-4b82-4f27-a00e-038b9b5185ba None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:19.455943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 568/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 09:59:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:19.461717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c786cd9-4ac9-4c09-8a10-268999c470b5 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:19.559011 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c786cd9-4ac9-4c09-8a10-268999c470b5 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:19.592215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b3ebe-415f-4c1f-b590-91bf1dab952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:19.592807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b3ebe-415f-4c1f-b590-91bf1dab952b None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:19.593117 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b3ebe-415f-4c1f-b590-91bf1dab952b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:19.593438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b3ebe-415f-4c1f-b590-91bf1dab952b None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:19.594339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b3ebe-415f-4c1f-b590-91bf1dab952b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:19.595756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b3ebe-415f-4c1f-b590-91bf1dab952b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:19.603505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:19.686219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 570/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:59:19 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:21.614529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70012c77-f719-41e0-85c4-a4ece338b1d4 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:21.615052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70012c77-f719-41e0-85c4-a4ece338b1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:21.615174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70012c77-f719-41e0-85c4-a4ece338b1d4 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:21.616447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 569/1139] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:21.621861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f66d7-5515-4c5a-b302-f0bf87991793 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:21.622296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f66d7-5515-4c5a-b302-f0bf87991793 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:21.622416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f66d7-5515-4c5a-b302-f0bf87991793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:21.641775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-fe7f66d7-5515-4c5a-b302-f0bf87991793 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:21.643108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-fe7f66d7-5515-4c5a-b302-f0bf87991793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:21.712471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 571/1140] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:21.982324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70012c77-f719-41e0-85c4-a4ece338b1d4 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:22.037754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:22.082149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70012c77-f719-41e0-85c4-a4ece338b1d4 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:22.115815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d75250-7144-4c3d-b880-6abbf549fa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:22.116429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d75250-7144-4c3d-b880-6abbf549fa84 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:22.116710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d75250-7144-4c3d-b880-6abbf549fa84 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:22.117012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d75250-7144-4c3d-b880-6abbf549fa84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:22.126836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d75250-7144-4c3d-b880-6abbf549fa84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-01d75250-7144-4c3d-b880-6abbf549fa84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:59:22.131503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:22.132626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:59:22.133607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 570/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:59:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:59:22.138640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f66d7-5515-4c5a-b302-f0bf87991793 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:22.232826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f66d7-5515-4c5a-b302-f0bf87991793 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:22.252210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:22.265433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87921365-9b89-4ac4-9553-c146793c6ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:22.266036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87921365-9b89-4ac4-9553-c146793c6ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:22.269207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87921365-9b89-4ac4-9553-c146793c6ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:22.269207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87921365-9b89-4ac4-9553-c146793c6ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:22.269207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87921365-9b89-4ac4-9553-c146793c6ad7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:22.269207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87921365-9b89-4ac4-9553-c146793c6ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:22.269555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87921365-9b89-4ac4-9553-c146793c6ad7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:22.277418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 572/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:59:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:22.284209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d75250-7144-4c3d-b880-6abbf549fa84 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:22.386281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d75250-7144-4c3d-b880-6abbf549fa84 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:22.421662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:22.441064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241e1d7-f14f-484c-b7da-53117313f5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:22.441683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241e1d7-f14f-484c-b7da-53117313f5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:22.441959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241e1d7-f14f-484c-b7da-53117313f5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:22.442227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241e1d7-f14f-484c-b7da-53117313f5f5 None admin] PATH_INFO: `/v3/users/e2e74f0dd6454be98154b773e061cf6f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:22.455498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241e1d7-f14f-484c-b7da-53117313f5f5 None admin] RBAC: Authorizing `identity:get_user(user_id=e2e74f0dd6454be98154b773e061cf6f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:22.456953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241e1d7-f14f-484c-b7da-53117313f5f5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:22.459008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 571/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:22 2022] GET /identity/v3/users/e2e74f0dd6454be98154b773e061cf6f => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:22.464739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87921365-9b89-4ac4-9553-c146793c6ad7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:22.563031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87921365-9b89-4ac4-9553-c146793c6ad7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:22.595897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806a0b4-16f4-4b29-bded-48b34759278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:22.596494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806a0b4-16f4-4b29-bded-48b34759278c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:22.596774 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806a0b4-16f4-4b29-bded-48b34759278c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:22.597076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806a0b4-16f4-4b29-bded-48b34759278c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:22.644939 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:22.665333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806a0b4-16f4-4b29-bded-48b34759278c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:22.666785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806a0b4-16f4-4b29-bded-48b34759278c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:22.764479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 573/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:59:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:59:22.771340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241e1d7-f14f-484c-b7da-53117313f5f5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:22.868981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241e1d7-f14f-484c-b7da-53117313f5f5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:22.902926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7387f9a-878d-4275-8109-081c8ecf9a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:22.903546 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7387f9a-878d-4275-8109-081c8ecf9a30 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:22.903827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7387f9a-878d-4275-8109-081c8ecf9a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:22.904103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7387f9a-878d-4275-8109-081c8ecf9a30 None admin] PATH_INFO: `/v3/projects/a0c05e36381e4953aa534998bc4a66fd` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:22.905244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7387f9a-878d-4275-8109-081c8ecf9a30 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c05e36381e4953aa534998bc4a66fd)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:22.906683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7387f9a-878d-4275-8109-081c8ecf9a30 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:22.908666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 572/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:59:22 2022] GET /identity/v3/projects/a0c05e36381e4953aa534998bc4a66fd => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:22.914896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806a0b4-16f4-4b29-bded-48b34759278c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:22.999343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:23.018890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806a0b4-16f4-4b29-bded-48b34759278c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:23.052912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a60d9b-045d-45e4-ae49-1bac4e969d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:23.053515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a60d9b-045d-45e4-ae49-1bac4e969d8d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:23.053798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a60d9b-045d-45e4-ae49-1bac4e969d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:23.054073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a60d9b-045d-45e4-ae49-1bac4e969d8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:23.054799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a60d9b-045d-45e4-ae49-1bac4e969d8d None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=a0c05e36381e4953aa534998bc4a66fd, user.id=e2e74f0dd6454be98154b773e061cf6f) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:23.055109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a60d9b-045d-45e4-ae49-1bac4e969d8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:23.056600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a60d9b-045d-45e4-ae49-1bac4e969d8d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:23.062347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 574/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:59:22 2022] GET /identity/v3/role_assignments?user.id=e2e74f0dd6454be98154b773e061cf6f&scope.project.id=a0c05e36381e4953aa534998bc4a66fd&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:24.957616 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61da8f6-6679-4683-9e3d-72823372f784 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:24.958208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61da8f6-6679-4683-9e3d-72823372f784 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:24.958208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61da8f6-6679-4683-9e3d-72823372f784 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:24.959486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 573/1147] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:24.964604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dbeb6-cd98-41e7-a733-4f1e0e0a28b5 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:24.965022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dbeb6-cd98-41e7-a733-4f1e0e0a28b5 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:24.965187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dbeb6-cd98-41e7-a733-4f1e0e0a28b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:24.983513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-629dbeb6-cd98-41e7-a733-4f1e0e0a28b5 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:24.984716 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-629dbeb6-cd98-41e7-a733-4f1e0e0a28b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:25.052224 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 575/1148] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:25.318787 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61da8f6-6679-4683-9e3d-72823372f784 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:25.361838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:25.418878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61da8f6-6679-4683-9e3d-72823372f784 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:25.451479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948fc82-a81f-4015-878d-eadc6fcb379f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:25.452130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948fc82-a81f-4015-878d-eadc6fcb379f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:25.452409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948fc82-a81f-4015-878d-eadc6fcb379f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:25.452688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948fc82-a81f-4015-878d-eadc6fcb379f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:25.463595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948fc82-a81f-4015-878d-eadc6fcb379f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-e948fc82-a81f-4015-878d-eadc6fcb379f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:59:25.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:25.470196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:25.471695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:59:25.472749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 574/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:59:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:59:25.476069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dbeb6-cd98-41e7-a733-4f1e0e0a28b5 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:25.550110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:25.572419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dbeb6-cd98-41e7-a733-4f1e0e0a28b5 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:25.606594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bd06d-f514-4868-b6c9-63ca166d64c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:25.607189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd06d-f514-4868-b6c9-63ca166d64c1 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:25.607472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd06d-f514-4868-b6c9-63ca166d64c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:25.607748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd06d-f514-4868-b6c9-63ca166d64c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:25.608406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bd06d-f514-4868-b6c9-63ca166d64c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:25.608712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bd06d-f514-4868-b6c9-63ca166d64c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:25.610143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bd06d-f514-4868-b6c9-63ca166d64c1 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:25.618079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 576/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:59:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:25.624065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948fc82-a81f-4015-878d-eadc6fcb379f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:25.719655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:25.722528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948fc82-a81f-4015-878d-eadc6fcb379f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:25.755649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ce48-d53f-40aa-8285-19d5ffed9f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:25.756260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ce48-d53f-40aa-8285-19d5ffed9f32 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:25.756537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ce48-d53f-40aa-8285-19d5ffed9f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:25.756810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ce48-d53f-40aa-8285-19d5ffed9f32 None admin] PATH_INFO: `/v3/users/e2e74f0dd6454be98154b773e061cf6f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:25.758083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838ce48-d53f-40aa-8285-19d5ffed9f32 None admin] RBAC: Authorizing `identity:get_user(user_id=e2e74f0dd6454be98154b773e061cf6f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:25.759497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838ce48-d53f-40aa-8285-19d5ffed9f32 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:25.761087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 575/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:25 2022] GET /identity/v3/users/e2e74f0dd6454be98154b773e061cf6f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:25.767679 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bd06d-f514-4868-b6c9-63ca166d64c1 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:25.869107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bd06d-f514-4868-b6c9-63ca166d64c1 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:25.902485 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a391be-11d3-4bcf-b5a3-4949a994ff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:25.903105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a391be-11d3-4bcf-b5a3-4949a994ff9a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:25.903388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a391be-11d3-4bcf-b5a3-4949a994ff9a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:25.903666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a391be-11d3-4bcf-b5a3-4949a994ff9a None admin] PATH_INFO: `/v3/projects/a0c05e36381e4953aa534998bc4a66fd` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:25.904773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a391be-11d3-4bcf-b5a3-4949a994ff9a None admin] RBAC: Authorizing `identity:get_project(project_id=a0c05e36381e4953aa534998bc4a66fd)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:25.906237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a391be-11d3-4bcf-b5a3-4949a994ff9a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:25.908222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 577/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:59:25 2022] GET /identity/v3/projects/a0c05e36381e4953aa534998bc4a66fd => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:25.914332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ce48-d53f-40aa-8285-19d5ffed9f32 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:26.012344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ce48-d53f-40aa-8285-19d5ffed9f32 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:26.045400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93ecc0-40d9-4a7d-9b85-c880f2568a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:26.046057 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93ecc0-40d9-4a7d-9b85-c880f2568a3d None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:26.046337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93ecc0-40d9-4a7d-9b85-c880f2568a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:26.046610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93ecc0-40d9-4a7d-9b85-c880f2568a3d None admin] PATH_INFO: `/v3/projects/a0c05e36381e4953aa534998bc4a66fd/users/e2e74f0dd6454be98154b773e061cf6f/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:26.049055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93ecc0-40d9-4a7d-9b85-c880f2568a3d None admin] RBAC: Authorizing `identity:create_grant(project_id=a0c05e36381e4953aa534998bc4a66fd, user_id=e2e74f0dd6454be98154b773e061cf6f, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:26.050508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93ecc0-40d9-4a7d-9b85-c880f2568a3d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:26.132546 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 576/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:59:25 2022] PUT /identity/v3/projects/a0c05e36381e4953aa534998bc4a66fd/users/e2e74f0dd6454be98154b773e061cf6f/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:28.004112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fb23f-cbc1-4f2d-9c73-a4f948bf1810 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:28.004665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fb23f-cbc1-4f2d-9c73-a4f948bf1810 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:28.004665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fb23f-cbc1-4f2d-9c73-a4f948bf1810 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:28.006045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 578/1154] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:28.011255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a7170-ef6d-44cf-bd36-935c66744dc8 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:28.011691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a7170-ef6d-44cf-bd36-935c66744dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:28.011796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a7170-ef6d-44cf-bd36-935c66744dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:28.030859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e14a7170-ef6d-44cf-bd36-935c66744dc8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:28.032089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e14a7170-ef6d-44cf-bd36-935c66744dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:28.054782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:28.100140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 577/1155] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:28.368721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fb23f-cbc1-4f2d-9c73-a4f948bf1810 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:28.379573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:28.468906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fb23f-cbc1-4f2d-9c73-a4f948bf1810 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:28.501758 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf0d6d-55b7-4b5e-80c9-8ecf4139ab90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:28.502389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf0d6d-55b7-4b5e-80c9-8ecf4139ab90 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:28.502874 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf0d6d-55b7-4b5e-80c9-8ecf4139ab90 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf0d6d-55b7-4b5e-80c9-8ecf4139ab90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf0d6d-55b7-4b5e-80c9-8ecf4139ab90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-95bf0d6d-55b7-4b5e-80c9-8ecf4139ab90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:28.524633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:28.525625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:28.526697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:59:28.527979 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 579/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:59:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:59:28.528072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a7170-ef6d-44cf-bd36-935c66744dc8 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:28.624675 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a7170-ef6d-44cf-bd36-935c66744dc8 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:28.658998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dbcf8-8e07-4905-a6e0-d583f2ca3977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:28.659594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dbcf8-8e07-4905-a6e0-d583f2ca3977 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:28.659877 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dbcf8-8e07-4905-a6e0-d583f2ca3977 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:28.660150 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dbcf8-8e07-4905-a6e0-d583f2ca3977 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:28.660807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dbcf8-8e07-4905-a6e0-d583f2ca3977 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:28.661138 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dbcf8-8e07-4905-a6e0-d583f2ca3977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:28.662537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dbcf8-8e07-4905-a6e0-d583f2ca3977 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:28.670518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 578/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:59:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:28.677461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf0d6d-55b7-4b5e-80c9-8ecf4139ab90 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:28.741971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:28.775249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf0d6d-55b7-4b5e-80c9-8ecf4139ab90 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:28.809247 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06dd28-c19f-4bf7-beaa-4858feb4a7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:28.809867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06dd28-c19f-4bf7-beaa-4858feb4a7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:28.810149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06dd28-c19f-4bf7-beaa-4858feb4a7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:28.810427 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06dd28-c19f-4bf7-beaa-4858feb4a7c0 None admin] PATH_INFO: `/v3/users/e2e74f0dd6454be98154b773e061cf6f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:28.811535 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06dd28-c19f-4bf7-beaa-4858feb4a7c0 None admin] RBAC: Authorizing `identity:get_user(user_id=e2e74f0dd6454be98154b773e061cf6f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:28.813127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06dd28-c19f-4bf7-beaa-4858feb4a7c0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:28.814843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 580/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:28 2022] GET /identity/v3/users/e2e74f0dd6454be98154b773e061cf6f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:28.820924 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dbcf8-8e07-4905-a6e0-d583f2ca3977 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:28.843556 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:28.919436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dbcf8-8e07-4905-a6e0-d583f2ca3977 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:28.953344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110953c-3b8f-47b2-aba5-27c736c2838f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:28.954622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110953c-3b8f-47b2-aba5-27c736c2838f None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:28.954836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110953c-3b8f-47b2-aba5-27c736c2838f None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:28.955117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110953c-3b8f-47b2-aba5-27c736c2838f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:29.024201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110953c-3b8f-47b2-aba5-27c736c2838f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:29.025688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110953c-3b8f-47b2-aba5-27c736c2838f None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:29.046468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:29.123107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 579/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:59:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:59:29.129141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06dd28-c19f-4bf7-beaa-4858feb4a7c0 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:29.228119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06dd28-c19f-4bf7-beaa-4858feb4a7c0 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:29.260066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955eb3f6-e674-4341-9343-ea455f638011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:29.260678 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eb3f6-e674-4341-9343-ea455f638011 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:29.260984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eb3f6-e674-4341-9343-ea455f638011 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:29.261261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eb3f6-e674-4341-9343-ea455f638011 None admin] PATH_INFO: `/v3/projects/a0c05e36381e4953aa534998bc4a66fd` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:29.262350 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955eb3f6-e674-4341-9343-ea455f638011 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c05e36381e4953aa534998bc4a66fd)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:29.263807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955eb3f6-e674-4341-9343-ea455f638011 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:29.265459 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 581/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:59:29 2022] GET /identity/v3/projects/a0c05e36381e4953aa534998bc4a66fd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:29.272169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110953c-3b8f-47b2-aba5-27c736c2838f None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:29.369777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110953c-3b8f-47b2-aba5-27c736c2838f None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:29.389710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:29.403587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec414851-e2ec-48ca-9828-3039ac6b355b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:29.404180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec414851-e2ec-48ca-9828-3039ac6b355b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:29.404455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec414851-e2ec-48ca-9828-3039ac6b355b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:29.404774 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec414851-e2ec-48ca-9828-3039ac6b355b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:29.405457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec414851-e2ec-48ca-9828-3039ac6b355b None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=a0c05e36381e4953aa534998bc4a66fd, user.id=e2e74f0dd6454be98154b773e061cf6f) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:29.405760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec414851-e2ec-48ca-9828-3039ac6b355b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:29.407201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec414851-e2ec-48ca-9828-3039ac6b355b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:29.412949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 580/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:59:29 2022] GET /identity/v3/role_assignments?user.id=e2e74f0dd6454be98154b773e061cf6f&scope.project.id=a0c05e36381e4953aa534998bc4a66fd&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:31.331939 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59376f6-cf15-457b-8722-60f3d73d7f23 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:31.332450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59376f6-cf15-457b-8722-60f3d73d7f23 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:31.332563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59376f6-cf15-457b-8722-60f3d73d7f23 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:31.333834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 582/1162] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:31.338957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6a798-42ae-4646-9d18-9eb4dddbbf97 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:31.339348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6a798-42ae-4646-9d18-9eb4dddbbf97 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:31.339515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6a798-42ae-4646-9d18-9eb4dddbbf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:31.357268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-7bf6a798-42ae-4646-9d18-9eb4dddbbf97 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:31.358448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7bf6a798-42ae-4646-9d18-9eb4dddbbf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:31.425303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 581/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:59:31.693173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59376f6-cf15-457b-8722-60f3d73d7f23 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:31.696039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:31.792760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59376f6-cf15-457b-8722-60f3d73d7f23 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:31.825140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0128764-c89e-42d2-a64d-eb4bb36285d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:31.825730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0128764-c89e-42d2-a64d-eb4bb36285d5 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:31.826005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0128764-c89e-42d2-a64d-eb4bb36285d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:31.826272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0128764-c89e-42d2-a64d-eb4bb36285d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:31.891973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0128764-c89e-42d2-a64d-eb4bb36285d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:31.893424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0128764-c89e-42d2-a64d-eb4bb36285d5 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:31.895162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:31.989170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 583/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:59:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:59:31.995242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6a798-42ae-4646-9d18-9eb4dddbbf97 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:32.092972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6a798-42ae-4646-9d18-9eb4dddbbf97 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:32.107262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:32.126622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651525fb-f2ec-46da-adb0-a83de2df6808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:32.127264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651525fb-f2ec-46da-adb0-a83de2df6808 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:32.127548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651525fb-f2ec-46da-adb0-a83de2df6808 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:32.127821 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651525fb-f2ec-46da-adb0-a83de2df6808 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:32.135238 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651525fb-f2ec-46da-adb0-a83de2df6808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:32.136732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651525fb-f2ec-46da-adb0-a83de2df6808 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-651525fb-f2ec-46da-adb0-a83de2df6808 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:32.149081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:32.150340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:59:32.151403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 582/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Aug 24 09:59:31 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:59:32.156414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0128764-c89e-42d2-a64d-eb4bb36285d5 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:32.255544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0128764-c89e-42d2-a64d-eb4bb36285d5 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:32.256958 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:32.290784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca1ec0-2321-4e00-952f-d95cb204fb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:32.291391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca1ec0-2321-4e00-952f-d95cb204fb9c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:32.291749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca1ec0-2321-4e00-952f-d95cb204fb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:32.292026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca1ec0-2321-4e00-952f-d95cb204fb9c None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:32.292657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca1ec0-2321-4e00-952f-d95cb204fb9c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:32.292991 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca1ec0-2321-4e00-952f-d95cb204fb9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:32.294352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca1ec0-2321-4e00-952f-d95cb204fb9c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:32.302401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 584/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Aug 24 09:59:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:34.193905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836bd06-8352-45eb-a275-856d674f1bc2 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:34.194453 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836bd06-8352-45eb-a275-856d674f1bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:34.194453 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836bd06-8352-45eb-a275-856d674f1bc2 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:34.195771 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 583/1167] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:34.201180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d5e1-f3fb-4d62-94be-83259b1c426e None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:34.201576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d5e1-f3fb-4d62-94be-83259b1c426e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:34.201733 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d5e1-f3fb-4d62-94be-83259b1c426e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:34.220600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-b0e1d5e1-f3fb-4d62-94be-83259b1c426e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:34.221816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b0e1d5e1-f3fb-4d62-94be-83259b1c426e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:34.287595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 585/1168] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:34.558172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836bd06-8352-45eb-a275-856d674f1bc2 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:34.658149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836bd06-8352-45eb-a275-856d674f1bc2 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:34.691386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b7c83-e4b6-408b-94d6-2b9ec362dd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:34.691975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b7c83-e4b6-408b-94d6-2b9ec362dd1e None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:34.692250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b7c83-e4b6-408b-94d6-2b9ec362dd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:34.692517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b7c83-e4b6-408b-94d6-2b9ec362dd1e None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:34.693198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b7c83-e4b6-408b-94d6-2b9ec362dd1e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:34.694557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b7c83-e4b6-408b-94d6-2b9ec362dd1e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:34.801422 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 584/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Wed Aug 24 09:59:34 2022] POST /identity/v3/domains => generated 267 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:36.778464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bb20d-2b57-4fcf-9719-42b97232f244 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:36.779022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bb20d-2b57-4fcf-9719-42b97232f244 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:36.779022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bb20d-2b57-4fcf-9719-42b97232f244 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:36.780350 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 586/1170] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:36.786118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255f619-8363-484d-b64d-bab874a560b7 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:36.786538 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255f619-8363-484d-b64d-bab874a560b7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:36.786669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255f619-8363-484d-b64d-bab874a560b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:36.805036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0255f619-8363-484d-b64d-bab874a560b7 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:36.806310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0255f619-8363-484d-b64d-bab874a560b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:36.833032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:36.871756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 585/1171] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:37.145323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bb20d-2b57-4fcf-9719-42b97232f244 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:37.238219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:37.244538 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bb20d-2b57-4fcf-9719-42b97232f244 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:37.279015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e3c1f-48f7-4711-a74f-7de187d5da47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:37.279911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e3c1f-48f7-4711-a74f-7de187d5da47 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:37.280195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e3c1f-48f7-4711-a74f-7de187d5da47 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:37.280474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e3c1f-48f7-4711-a74f-7de187d5da47 None admin] PATH_INFO: `/v3/domains/eb020d021e6344a8b018dae0fa04d673` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:37.291881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e3c1f-48f7-4711-a74f-7de187d5da47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=eb020d021e6344a8b018dae0fa04d673)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:37.293367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e3c1f-48f7-4711-a74f-7de187d5da47 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:37.295030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 587/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Wed Aug 24 09:59:37 2022] GET /identity/v3/domains/eb020d021e6344a8b018dae0fa04d673 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:37.300984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255f619-8363-484d-b64d-bab874a560b7 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:37.397531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255f619-8363-484d-b64d-bab874a560b7 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:37.430901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50279f00-0b70-4581-aaef-455f859f0a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:37.431497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50279f00-0b70-4581-aaef-455f859f0a69 None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:37.431776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50279f00-0b70-4581-aaef-455f859f0a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:37.432050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50279f00-0b70-4581-aaef-455f859f0a69 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:37.432678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50279f00-0b70-4581-aaef-455f859f0a69 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:37.434133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50279f00-0b70-4581-aaef-455f859f0a69 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:37.524914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 586/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 09:59:37 2022] POST /identity/v3/projects => generated 361 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:39.458296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1db770-0d9d-4ff4-85e7-29c3394b62f0 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.458833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1db770-0d9d-4ff4-85e7-29c3394b62f0 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.458833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1db770-0d9d-4ff4-85e7-29c3394b62f0 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.460199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 588/1174] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:39.465570 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cc330-fc8e-4a37-bfc6-96cb8fa0dd2b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.465981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cc330-fc8e-4a37-bfc6-96cb8fa0dd2b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.466124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cc330-fc8e-4a37-bfc6-96cb8fa0dd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.483753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3d9cc330-fc8e-4a37-bfc6-96cb8fa0dd2b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:39.485637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3d9cc330-fc8e-4a37-bfc6-96cb8fa0dd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:39.496150 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:39.550774 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 587/1175] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:39.819122 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1db770-0d9d-4ff4-85e7-29c3394b62f0 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.920007 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1db770-0d9d-4ff4-85e7-29c3394b62f0 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.952580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a969f-cd1b-4c99-8c2c-f3b8f4b4cc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.953221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a969f-cd1b-4c99-8c2c-f3b8f4b4cc23 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.954143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a969f-cd1b-4c99-8c2c-f3b8f4b4cc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.954429 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a969f-cd1b-4c99-8c2c-f3b8f4b4cc23 None admin] PATH_INFO: `/v3/domains/eb020d021e6344a8b018dae0fa04d673` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.955542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a969f-cd1b-4c99-8c2c-f3b8f4b4cc23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=eb020d021e6344a8b018dae0fa04d673)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.957010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a969f-cd1b-4c99-8c2c-f3b8f4b4cc23 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.958563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 589/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Wed Aug 24 09:59:39 2022] GET /identity/v3/domains/eb020d021e6344a8b018dae0fa04d673 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:39.964320 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9cc330-fc8e-4a37-bfc6-96cb8fa0dd2b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:40.061787 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9cc330-fc8e-4a37-bfc6-96cb8fa0dd2b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:40.095773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfda3b-7662-4047-b11d-ce60ca5064db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:40.096298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfda3b-7662-4047-b11d-ce60ca5064db None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:40.096574 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfda3b-7662-4047-b11d-ce60ca5064db None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:40.096846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfda3b-7662-4047-b11d-ce60ca5064db None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:40.097513 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfda3b-7662-4047-b11d-ce60ca5064db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:40.098918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfda3b-7662-4047-b11d-ce60ca5064db None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:40.216830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 588/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/users => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:42.169016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1d06f-1bd8-491c-9398-1dff9588542d None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:42.169548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1d06f-1bd8-491c-9398-1dff9588542d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:42.169548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1d06f-1bd8-491c-9398-1dff9588542d None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:42.170909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 590/1178] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:42.176758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac30cb-60c6-4d06-83a2-08543a5224a7 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:42.177241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac30cb-60c6-4d06-83a2-08543a5224a7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:42.177343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac30cb-60c6-4d06-83a2-08543a5224a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:42.183480 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:42.195208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-54ac30cb-60c6-4d06-83a2-08543a5224a7 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:42.196371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-54ac30cb-60c6-4d06-83a2-08543a5224a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:42.261213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 589/1179] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:59:42.534044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1d06f-1bd8-491c-9398-1dff9588542d None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:42.559046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:42.633605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1d06f-1bd8-491c-9398-1dff9588542d None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:42.666967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bcd33-a1cd-45b9-9592-c3abaf624f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:42.667585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bcd33-a1cd-45b9-9592-c3abaf624f9c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:42.667867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bcd33-a1cd-45b9-9592-c3abaf624f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:42.668143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bcd33-a1cd-45b9-9592-c3abaf624f9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:42.678598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bcd33-a1cd-45b9-9592-c3abaf624f9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-873bcd33-a1cd-45b9-9592-c3abaf624f9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:59:42.683277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:42.684399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:59:42.685738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 591/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:59:42.690679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac30cb-60c6-4d06-83a2-08543a5224a7 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:42.788549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac30cb-60c6-4d06-83a2-08543a5224a7 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:42.817120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:42.821939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234498ae-2749-4f77-98bd-fc3f994dc663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:42.822543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234498ae-2749-4f77-98bd-fc3f994dc663 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:42.822827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234498ae-2749-4f77-98bd-fc3f994dc663 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:42.823102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234498ae-2749-4f77-98bd-fc3f994dc663 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:42.823769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234498ae-2749-4f77-98bd-fc3f994dc663 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:42.824071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234498ae-2749-4f77-98bd-fc3f994dc663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:42.825488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234498ae-2749-4f77-98bd-fc3f994dc663 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:42.834356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 590/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:59:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:42.840441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bcd33-a1cd-45b9-9592-c3abaf624f9c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:42.924393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:42.941053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bcd33-a1cd-45b9-9592-c3abaf624f9c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:42.973967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb799e39-95d7-4c38-aac7-3778d8f239f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:42.974575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799e39-95d7-4c38-aac7-3778d8f239f3 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:42.974852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799e39-95d7-4c38-aac7-3778d8f239f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:42.975119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799e39-95d7-4c38-aac7-3778d8f239f3 None admin] PATH_INFO: `/v3/users/44d61a6af46f4846bfcc2f1a43399b17` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:42.987956 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb799e39-95d7-4c38-aac7-3778d8f239f3 None admin] RBAC: Authorizing `identity:get_user(user_id=44d61a6af46f4846bfcc2f1a43399b17)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:42.989411 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb799e39-95d7-4c38-aac7-3778d8f239f3 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:42.990973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 592/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:42 2022] GET /identity/v3/users/44d61a6af46f4846bfcc2f1a43399b17 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:42.997135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234498ae-2749-4f77-98bd-fc3f994dc663 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:43.093542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234498ae-2749-4f77-98bd-fc3f994dc663 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:43.125085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a111532-e8de-4e83-b5fc-05a583b2d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:43.126225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a111532-e8de-4e83-b5fc-05a583b2d725 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:43.126495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a111532-e8de-4e83-b5fc-05a583b2d725 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:43.126763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a111532-e8de-4e83-b5fc-05a583b2d725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:43.184703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:43.192693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a111532-e8de-4e83-b5fc-05a583b2d725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:43.197942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a111532-e8de-4e83-b5fc-05a583b2d725 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:43.289810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 591/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:59:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:59:43.295763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb799e39-95d7-4c38-aac7-3778d8f239f3 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:43.393402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb799e39-95d7-4c38-aac7-3778d8f239f3 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:43.426809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fdd99-c229-492e-82c3-b59297d88fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:43.427426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fdd99-c229-492e-82c3-b59297d88fbc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:43.427707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fdd99-c229-492e-82c3-b59297d88fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:43.427983 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fdd99-c229-492e-82c3-b59297d88fbc None admin] PATH_INFO: `/v3/projects/623c4d153a7a4392b12ae20afd35aa55` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:43.429117 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fdd99-c229-492e-82c3-b59297d88fbc None admin] RBAC: Authorizing `identity:get_project(project_id=623c4d153a7a4392b12ae20afd35aa55)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:43.430543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fdd99-c229-492e-82c3-b59297d88fbc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:43.432151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 593/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:59:43 2022] GET /identity/v3/projects/623c4d153a7a4392b12ae20afd35aa55 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:43.439052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a111532-e8de-4e83-b5fc-05a583b2d725 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:43.529465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:43.536070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a111532-e8de-4e83-b5fc-05a583b2d725 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:43.570112 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64017573-649c-4cb5-9f5a-d937d72683c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:43.570706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64017573-649c-4cb5-9f5a-d937d72683c8 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:43.571020 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64017573-649c-4cb5-9f5a-d937d72683c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:43.571323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64017573-649c-4cb5-9f5a-d937d72683c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:43.572074 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64017573-649c-4cb5-9f5a-d937d72683c8 None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=623c4d153a7a4392b12ae20afd35aa55, user.id=44d61a6af46f4846bfcc2f1a43399b17) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:43.572395 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64017573-649c-4cb5-9f5a-d937d72683c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:43.573867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64017573-649c-4cb5-9f5a-d937d72683c8 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:43.579540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 592/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:59:43 2022] GET /identity/v3/role_assignments?user.id=44d61a6af46f4846bfcc2f1a43399b17&scope.project.id=623c4d153a7a4392b12ae20afd35aa55&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:45.471353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbfc71-bf31-4e2b-9524-dd8382dca44c None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:45.471902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbfc71-bf31-4e2b-9524-dd8382dca44c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:45.471902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbfc71-bf31-4e2b-9524-dd8382dca44c None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:45.473300 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 594/1186] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:45.478559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae6716-56d2-4351-8a3a-b89758fde307 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:45.478963 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae6716-56d2-4351-8a3a-b89758fde307 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:45.479113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae6716-56d2-4351-8a3a-b89758fde307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:45.496646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3fae6716-56d2-4351-8a3a-b89758fde307 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:45.497856 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3fae6716-56d2-4351-8a3a-b89758fde307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:45.562061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 593/1187] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:45.834098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbfc71-bf31-4e2b-9524-dd8382dca44c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:45.847087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:45.939532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbfc71-bf31-4e2b-9524-dd8382dca44c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:45.975725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6b29d-6c61-4142-95d2-63341a64d125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:45.976352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6b29d-6c61-4142-95d2-63341a64d125 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:45.976633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6b29d-6c61-4142-95d2-63341a64d125 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:45.976932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6b29d-6c61-4142-95d2-63341a64d125 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:45.987323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6b29d-6c61-4142-95d2-63341a64d125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-70d6b29d-6c61-4142-95d2-63341a64d125 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:59:45.992052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:45.993295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 09:59:45.994174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 595/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:59:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:59:45.999458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae6716-56d2-4351-8a3a-b89758fde307 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:46.094359 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:46.106653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae6716-56d2-4351-8a3a-b89758fde307 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:46.146137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e00f3-994d-469d-af1d-ecbbeba36743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:46.146754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e00f3-994d-469d-af1d-ecbbeba36743 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:46.147054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e00f3-994d-469d-af1d-ecbbeba36743 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:46.147332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e00f3-994d-469d-af1d-ecbbeba36743 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:46.148004 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e00f3-994d-469d-af1d-ecbbeba36743 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:46.148314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e00f3-994d-469d-af1d-ecbbeba36743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:46.149728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e00f3-994d-469d-af1d-ecbbeba36743 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:46.159019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 594/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:59:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:46.164965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6b29d-6c61-4142-95d2-63341a64d125 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:46.235871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:46.266141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6b29d-6c61-4142-95d2-63341a64d125 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:46.303909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b31d01-058c-4855-8992-cbc9affee47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:46.304536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b31d01-058c-4855-8992-cbc9affee47b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:46.304819 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b31d01-058c-4855-8992-cbc9affee47b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:46.305125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b31d01-058c-4855-8992-cbc9affee47b None admin] PATH_INFO: `/v3/users/44d61a6af46f4846bfcc2f1a43399b17` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:46.306264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b31d01-058c-4855-8992-cbc9affee47b None admin] RBAC: Authorizing `identity:get_user(user_id=44d61a6af46f4846bfcc2f1a43399b17)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:46.307685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b31d01-058c-4855-8992-cbc9affee47b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:46.309315 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 596/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:46 2022] GET /identity/v3/users/44d61a6af46f4846bfcc2f1a43399b17 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:46.316212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e00f3-994d-469d-af1d-ecbbeba36743 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:46.423851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e00f3-994d-469d-af1d-ecbbeba36743 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:46.458943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d907f7-869b-4dda-953a-cbc60c68bf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:46.459569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d907f7-869b-4dda-953a-cbc60c68bf8b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:46.459846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d907f7-869b-4dda-953a-cbc60c68bf8b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:46.460112 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d907f7-869b-4dda-953a-cbc60c68bf8b None admin] PATH_INFO: `/v3/projects/623c4d153a7a4392b12ae20afd35aa55` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:46.461238 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d907f7-869b-4dda-953a-cbc60c68bf8b None admin] RBAC: Authorizing `identity:get_project(project_id=623c4d153a7a4392b12ae20afd35aa55)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:46.462662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d907f7-869b-4dda-953a-cbc60c68bf8b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:46.464287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 595/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:59:46 2022] GET /identity/v3/projects/623c4d153a7a4392b12ae20afd35aa55 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:46.470622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b31d01-058c-4855-8992-cbc9affee47b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:46.570627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b31d01-058c-4855-8992-cbc9affee47b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:46.604472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828409c9-c182-402d-8f7a-534ddcb98b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:46.605145 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828409c9-c182-402d-8f7a-534ddcb98b8b None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:46.605431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828409c9-c182-402d-8f7a-534ddcb98b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:46.605708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828409c9-c182-402d-8f7a-534ddcb98b8b None admin] PATH_INFO: `/v3/projects/623c4d153a7a4392b12ae20afd35aa55/users/44d61a6af46f4846bfcc2f1a43399b17/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:46.607585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828409c9-c182-402d-8f7a-534ddcb98b8b None admin] RBAC: Authorizing `identity:create_grant(project_id=623c4d153a7a4392b12ae20afd35aa55, user_id=44d61a6af46f4846bfcc2f1a43399b17, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:46.609039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828409c9-c182-402d-8f7a-534ddcb98b8b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:46.674929 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:46.680222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 597/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 09:59:46 2022] PUT /identity/v3/projects/623c4d153a7a4392b12ae20afd35aa55/users/44d61a6af46f4846bfcc2f1a43399b17/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:59:48.571178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a2b0-ee2a-4137-95f8-86e79b7f5a98 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:48.571677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a2b0-ee2a-4137-95f8-86e79b7f5a98 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:48.571677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a2b0-ee2a-4137-95f8-86e79b7f5a98 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:48.573038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 596/1193] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 09:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:48.578271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28f211-e986-4f4c-8162-9fd7899abbad None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:48.578879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28f211-e986-4f4c-8162-9fd7899abbad None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:48.578879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28f211-e986-4f4c-8162-9fd7899abbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:48.596459 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-cd28f211-e986-4f4c-8162-9fd7899abbad None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:48.597683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-cd28f211-e986-4f4c-8162-9fd7899abbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:48.662987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 598/1194] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 09:59:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:59:48.938833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795a2b0-ee2a-4137-95f8-86e79b7f5a98 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:48.958444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:49.041593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795a2b0-ee2a-4137-95f8-86e79b7f5a98 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:49.074312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3eec2-a206-48bf-900b-b953c750ee2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:49.074929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3eec2-a206-48bf-900b-b953c750ee2d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:49.075212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3eec2-a206-48bf-900b-b953c750ee2d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:49.075489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3eec2-a206-48bf-900b-b953c750ee2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:49.085825 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3eec2-a206-48bf-900b-b953c750ee2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-c9a3eec2-a206-48bf-900b-b953c750ee2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:59:49.090549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:59:49.091697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 09:59:49.093065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 597/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 09:59:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:59:49.098390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28f211-e986-4f4c-8162-9fd7899abbad None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:49.195690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28f211-e986-4f4c-8162-9fd7899abbad None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:49.221894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:49.229313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5ef1a-3015-433c-bf95-f119a7df70a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:49.229901 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ef1a-3015-433c-bf95-f119a7df70a0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:49.230177 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ef1a-3015-433c-bf95-f119a7df70a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:49.230443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ef1a-3015-433c-bf95-f119a7df70a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:49.231123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5ef1a-3015-433c-bf95-f119a7df70a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:49.231419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5ef1a-3015-433c-bf95-f119a7df70a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:49.232842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5ef1a-3015-433c-bf95-f119a7df70a0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:49.240850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 599/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 09:59:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:49.246769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3eec2-a206-48bf-900b-b953c750ee2d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:49.325116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:49.347141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3eec2-a206-48bf-900b-b953c750ee2d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:49.380268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332047-a6b5-4bb3-a558-c6eb62d6270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:49.380908 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332047-a6b5-4bb3-a558-c6eb62d6270a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:49.381192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332047-a6b5-4bb3-a558-c6eb62d6270a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:49.381465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332047-a6b5-4bb3-a558-c6eb62d6270a None admin] PATH_INFO: `/v3/users/44d61a6af46f4846bfcc2f1a43399b17` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:49.382562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b332047-a6b5-4bb3-a558-c6eb62d6270a None admin] RBAC: Authorizing `identity:get_user(user_id=44d61a6af46f4846bfcc2f1a43399b17)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:49.383978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b332047-a6b5-4bb3-a558-c6eb62d6270a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:49.385578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 598/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Aug 24 09:59:49 2022] GET /identity/v3/users/44d61a6af46f4846bfcc2f1a43399b17 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:49.394043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5ef1a-3015-433c-bf95-f119a7df70a0 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:49.493205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5ef1a-3015-433c-bf95-f119a7df70a0 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:49.526405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b4be1-cfe8-4aee-b1c4-3b0db1110a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:49.527026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b4be1-cfe8-4aee-b1c4-3b0db1110a69 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:49.527305 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b4be1-cfe8-4aee-b1c4-3b0db1110a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:49.527581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b4be1-cfe8-4aee-b1c4-3b0db1110a69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:49.584635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:49.595415 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b4be1-cfe8-4aee-b1c4-3b0db1110a69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:49.596840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b4be1-cfe8-4aee-b1c4-3b0db1110a69 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:49.692999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 600/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 09:59:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:59:49.698912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332047-a6b5-4bb3-a558-c6eb62d6270a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:49.804155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332047-a6b5-4bb3-a558-c6eb62d6270a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:49.837822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c413eaf-2c54-4602-9ef8-d873bcb756b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:49.838462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c413eaf-2c54-4602-9ef8-d873bcb756b1 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:49.838744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c413eaf-2c54-4602-9ef8-d873bcb756b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:49.839018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c413eaf-2c54-4602-9ef8-d873bcb756b1 None admin] PATH_INFO: `/v3/projects/623c4d153a7a4392b12ae20afd35aa55` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:49.840124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c413eaf-2c54-4602-9ef8-d873bcb756b1 None admin] RBAC: Authorizing `identity:get_project(project_id=623c4d153a7a4392b12ae20afd35aa55)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:49.841587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c413eaf-2c54-4602-9ef8-d873bcb756b1 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:49.843200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 599/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Aug 24 09:59:49 2022] GET /identity/v3/projects/623c4d153a7a4392b12ae20afd35aa55 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:49.849544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b4be1-cfe8-4aee-b1c4-3b0db1110a69 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:49.936084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:49.945682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b4be1-cfe8-4aee-b1c4-3b0db1110a69 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:49.978914 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780274f4-5baf-44e7-bf60-5308b959dde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:49.979503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780274f4-5baf-44e7-bf60-5308b959dde9 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:49.979785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780274f4-5baf-44e7-bf60-5308b959dde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:49.980059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780274f4-5baf-44e7-bf60-5308b959dde9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:49.980775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780274f4-5baf-44e7-bf60-5308b959dde9 None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=623c4d153a7a4392b12ae20afd35aa55, user.id=44d61a6af46f4846bfcc2f1a43399b17) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:49.981113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780274f4-5baf-44e7-bf60-5308b959dde9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:49.982524 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780274f4-5baf-44e7-bf60-5308b959dde9 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:49.988785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 601/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 09:59:49 2022] GET /identity/v3/role_assignments?user.id=44d61a6af46f4846bfcc2f1a43399b17&scope.project.id=623c4d153a7a4392b12ae20afd35aa55&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:59.833694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=93 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:59.986206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=23 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:05.421784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ecb1a-58f7-4df5-b6a2-79f969ce5f59 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:05.422382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ecb1a-58f7-4df5-b6a2-79f969ce5f59 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:05.422382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ecb1a-58f7-4df5-b6a2-79f969ce5f59 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:05.423971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 600/1201] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:05.429284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eee5ea-d53d-47d1-994f-51e97786d740 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:05.429751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eee5ea-d53d-47d1-994f-51e97786d740 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:05.429851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eee5ea-d53d-47d1-994f-51e97786d740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:05.447550 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-08eee5ea-d53d-47d1-994f-51e97786d740 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:05.448714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-08eee5ea-d53d-47d1-994f-51e97786d740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:05.514147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 602/1202] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:01:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:01:05.788406 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ecb1a-58f7-4df5-b6a2-79f969ce5f59 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:05.891670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ecb1a-58f7-4df5-b6a2-79f969ce5f59 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:05.924586 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02ac4d-aa10-418a-9492-f9e56e088167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:05.929137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02ac4d-aa10-418a-9492-f9e56e088167 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:05.929137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02ac4d-aa10-418a-9492-f9e56e088167 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:05.929137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02ac4d-aa10-418a-9492-f9e56e088167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:05.935864 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02ac4d-aa10-418a-9492-f9e56e088167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:05.937311 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02ac4d-aa10-418a-9492-f9e56e088167 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-4e02ac4d-aa10-418a-9492-f9e56e088167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:05.950891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:05.952726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:05.954117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:05.954117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:05.954117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:05.954117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 601/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:05.958585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eee5ea-d53d-47d1-994f-51e97786d740 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:06.054227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eee5ea-d53d-47d1-994f-51e97786d740 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:06.083285 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:06.088951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd463f26-04c3-4a7c-ba4f-43c43e2790cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:06.089549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd463f26-04c3-4a7c-ba4f-43c43e2790cf None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:06.089826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd463f26-04c3-4a7c-ba4f-43c43e2790cf None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:06.090095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd463f26-04c3-4a7c-ba4f-43c43e2790cf None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:06.090721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd463f26-04c3-4a7c-ba4f-43c43e2790cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:06.091017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd463f26-04c3-4a7c-ba4f-43c43e2790cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:06.092511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd463f26-04c3-4a7c-ba4f-43c43e2790cf None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:06.103379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 603/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:06.109328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02ac4d-aa10-418a-9492-f9e56e088167 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:06.163322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:06.209728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02ac4d-aa10-418a-9492-f9e56e088167 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:06.243176 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae63bf-98b2-4b08-b7e4-e591db16bbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:06.243781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae63bf-98b2-4b08-b7e4-e591db16bbf7 None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:06.244061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae63bf-98b2-4b08-b7e4-e591db16bbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:06.244361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae63bf-98b2-4b08-b7e4-e591db16bbf7 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:06.245030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae63bf-98b2-4b08-b7e4-e591db16bbf7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:06.246503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae63bf-98b2-4b08-b7e4-e591db16bbf7 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:06.327890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 602/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Aug 24 10:01:06 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:08.295521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3903600-d8b5-4c06-90a6-db408a6cbe93 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:08.296625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3903600-d8b5-4c06-90a6-db408a6cbe93 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:08.296625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3903600-d8b5-4c06-90a6-db408a6cbe93 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:08.297621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 604/1206] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:08.302934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8a1b0-f7a9-4316-bbae-050dbdd1649d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:08.304988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8a1b0-f7a9-4316-bbae-050dbdd1649d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:08.304988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8a1b0-f7a9-4316-bbae-050dbdd1649d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:08.321447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-1db8a1b0-f7a9-4316-bbae-050dbdd1649d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:08.322620 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-1db8a1b0-f7a9-4316-bbae-050dbdd1649d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:08.390396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 603/1207] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:01:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:01:08.664602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3903600-d8b5-4c06-90a6-db408a6cbe93 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:08.770641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3903600-d8b5-4c06-90a6-db408a6cbe93 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:08.805877 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87797df0-0fb1-4ab8-a61b-dbe589f1fca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:08.806494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87797df0-0fb1-4ab8-a61b-dbe589f1fca5 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:08.806773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87797df0-0fb1-4ab8-a61b-dbe589f1fca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:08.807042 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87797df0-0fb1-4ab8-a61b-dbe589f1fca5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:08.817959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87797df0-0fb1-4ab8-a61b-dbe589f1fca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-87797df0-0fb1-4ab8-a61b-dbe589f1fca5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:01:08.822934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:08.824113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:08.825151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 605/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 10:01:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:08.830259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8a1b0-f7a9-4316-bbae-050dbdd1649d None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:08.872849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:08.932462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8a1b0-f7a9-4316-bbae-050dbdd1649d None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:08.967416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75542091-4f9f-4143-b391-32325ca8b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:08.968013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75542091-4f9f-4143-b391-32325ca8b1db None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:08.968290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75542091-4f9f-4143-b391-32325ca8b1db None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:08.968557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75542091-4f9f-4143-b391-32325ca8b1db None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:08.969237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75542091-4f9f-4143-b391-32325ca8b1db None admin] RBAC: Adding query filter params (name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:08.969533 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75542091-4f9f-4143-b391-32325ca8b1db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:08.970887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75542091-4f9f-4143-b391-32325ca8b1db None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:08.978495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 604/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 10:01:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:08.985039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87797df0-0fb1-4ab8-a61b-dbe589f1fca5 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:09.038530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:09.087700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87797df0-0fb1-4ab8-a61b-dbe589f1fca5 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:09.122820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a2c60-61e4-4536-aea4-fd84deb26f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:09.123440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a2c60-61e4-4536-aea4-fd84deb26f4e None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:09.123721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a2c60-61e4-4536-aea4-fd84deb26f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:09.123993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a2c60-61e4-4536-aea4-fd84deb26f4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:09.134505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a2c60-61e4-4536-aea4-fd84deb26f4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:09.135940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a2c60-61e4-4536-aea4-fd84deb26f4e None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-135a2c60-61e4-4536-aea4-fd84deb26f4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:09.149829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:09.156383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:09.157651 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:09.157651 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:09.157651 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:09.157651 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 606/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:09.157936 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75542091-4f9f-4143-b391-32325ca8b1db None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:09.256010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:09.258846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75542091-4f9f-4143-b391-32325ca8b1db None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:09.292744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aab057-441c-4db8-9530-59671df04565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:09.293410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aab057-441c-4db8-9530-59671df04565 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:09.293690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aab057-441c-4db8-9530-59671df04565 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:09.293984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aab057-441c-4db8-9530-59671df04565 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:09.294638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aab057-441c-4db8-9530-59671df04565 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:09.295481 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aab057-441c-4db8-9530-59671df04565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:09.296847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aab057-441c-4db8-9530-59671df04565 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:09.308565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 605/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:01:09.314489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a2c60-61e4-4536-aea4-fd84deb26f4e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:09.424442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a2c60-61e4-4536-aea4-fd84deb26f4e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:09.426078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:09.458980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5354e-72ff-4cbe-810e-00533b332ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:09.459611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5354e-72ff-4cbe-810e-00533b332ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:09.459893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5354e-72ff-4cbe-810e-00533b332ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:09.460171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5354e-72ff-4cbe-810e-00533b332ce3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:09.468346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5354e-72ff-4cbe-810e-00533b332ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:09.469904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5354e-72ff-4cbe-810e-00533b332ce3 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-b1f5354e-72ff-4cbe-810e-00533b332ce3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:01:09.480735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:09.481896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:09.483512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:09.483512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:09.483512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:01:09.483512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:01:09.483512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:01:09.483512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:01:09.483512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:01:09.483512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:01:09.483512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:09.483512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:09.483512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 607/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:09.488055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aab057-441c-4db8-9530-59671df04565 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:09.587862 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aab057-441c-4db8-9530-59671df04565 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:09.623306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73810f6b-d289-4e6c-bdfa-d8a8f15d9d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:09.623997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73810f6b-d289-4e6c-bdfa-d8a8f15d9d62 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:09.624279 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73810f6b-d289-4e6c-bdfa-d8a8f15d9d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:09.624553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73810f6b-d289-4e6c-bdfa-d8a8f15d9d62 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:09.625321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73810f6b-d289-4e6c-bdfa-d8a8f15d9d62 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:09.625629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73810f6b-d289-4e6c-bdfa-d8a8f15d9d62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:09.627055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73810f6b-d289-4e6c-bdfa-d8a8f15d9d62 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:09.645211 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 606/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Aug 24 10:01:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:09.651292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5354e-72ff-4cbe-810e-00533b332ce3 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:09.753760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5354e-72ff-4cbe-810e-00533b332ce3 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:09.788389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8544b-bf36-4a4d-8f6f-89fe8b4df47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:09.789015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8544b-bf36-4a4d-8f6f-89fe8b4df47d None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:09.789294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8544b-bf36-4a4d-8f6f-89fe8b4df47d None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:09.789583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8544b-bf36-4a4d-8f6f-89fe8b4df47d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:09.819664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:09.856017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8544b-bf36-4a4d-8f6f-89fe8b4df47d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:09.857850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8544b-bf36-4a4d-8f6f-89fe8b4df47d None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:09.955111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 608/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 10:01:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:01:09.960950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73810f6b-d289-4e6c-bdfa-d8a8f15d9d62 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:09.980078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:10.059142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73810f6b-d289-4e6c-bdfa-d8a8f15d9d62 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:10.094820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6f7ef-bda9-4461-a7f7-9a644be8615a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:10.095446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f7ef-bda9-4461-a7f7-9a644be8615a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:10.095725 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f7ef-bda9-4461-a7f7-9a644be8615a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:10.095999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f7ef-bda9-4461-a7f7-9a644be8615a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:10.106644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6f7ef-bda9-4461-a7f7-9a644be8615a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:10.108038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6f7ef-bda9-4461-a7f7-9a644be8615a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-99e6f7ef-bda9-4461-a7f7-9a644be8615a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:10.122400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:10.124242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:10.125642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:10.125642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:10.125642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:10.125642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 607/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:10.130141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8544b-bf36-4a4d-8f6f-89fe8b4df47d None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:10.193873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:10.230801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8544b-bf36-4a4d-8f6f-89fe8b4df47d None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:10.266428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f190349-a6f0-401e-bda0-1284391a2b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:10.267089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f190349-a6f0-401e-bda0-1284391a2b0a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:10.267369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f190349-a6f0-401e-bda0-1284391a2b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:10.267639 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f190349-a6f0-401e-bda0-1284391a2b0a None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:10.268338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f190349-a6f0-401e-bda0-1284391a2b0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:10.268642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f190349-a6f0-401e-bda0-1284391a2b0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:10.270195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f190349-a6f0-401e-bda0-1284391a2b0a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:10.282559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 609/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:01:10.288267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6f7ef-bda9-4461-a7f7-9a644be8615a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:10.358722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:10.389342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6f7ef-bda9-4461-a7f7-9a644be8615a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:10.422414 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33540393-cfe6-4d17-89d0-81b885f04aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:10.423038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33540393-cfe6-4d17-89d0-81b885f04aec None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:10.423319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33540393-cfe6-4d17-89d0-81b885f04aec None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:10.423593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33540393-cfe6-4d17-89d0-81b885f04aec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:10.431191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33540393-cfe6-4d17-89d0-81b885f04aec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:10.436679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33540393-cfe6-4d17-89d0-81b885f04aec None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-33540393-cfe6-4d17-89d0-81b885f04aec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:01:10.444806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:10.445912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:10.447223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:01:10.447223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:01:10.447223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:01:10.447223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:01:10.447223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:01:10.447223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:01:10.447223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:10.447223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:10.447223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 608/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 10:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:10.451884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f190349-a6f0-401e-bda0-1284391a2b0a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:10.552507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f190349-a6f0-401e-bda0-1284391a2b0a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:10.574756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:10.585941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81379463-704f-4b0b-a2bc-e4273de949bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:10.586534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81379463-704f-4b0b-a2bc-e4273de949bd None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:10.586814 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81379463-704f-4b0b-a2bc-e4273de949bd None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:10.587090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81379463-704f-4b0b-a2bc-e4273de949bd None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:10.587770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81379463-704f-4b0b-a2bc-e4273de949bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:10.588073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81379463-704f-4b0b-a2bc-e4273de949bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:10.589645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81379463-704f-4b0b-a2bc-e4273de949bd None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:10.602389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 610/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 10:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:10.609404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33540393-cfe6-4d17-89d0-81b885f04aec None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:10.712280 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33540393-cfe6-4d17-89d0-81b885f04aec None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:10.741755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:10.747389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ecf8d-0df1-43f7-8155-f08ec00699e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:10.748013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ecf8d-0df1-43f7-8155-f08ec00699e9 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:10.748281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ecf8d-0df1-43f7-8155-f08ec00699e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:10.748547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ecf8d-0df1-43f7-8155-f08ec00699e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:10.749296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ecf8d-0df1-43f7-8155-f08ec00699e9 None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=b1aa47a883604336a8ab90d324470477) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:10.749603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ecf8d-0df1-43f7-8155-f08ec00699e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:10.751031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ecf8d-0df1-43f7-8155-f08ec00699e9 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:10.758109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 609/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 10:01:10 2022] GET /identity/v3/role_assignments?user.id=b1aa47a883604336a8ab90d324470477&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:12.680132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7afc0-bc3d-4612-ad5e-02d103238f1c None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:12.680655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7afc0-bc3d-4612-ad5e-02d103238f1c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:12.680778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7afc0-bc3d-4612-ad5e-02d103238f1c None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:12.682125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 611/1220] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:12.688187 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a9f57-f20a-4e15-b0b5-ad15217e72e3 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:12.688611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a9f57-f20a-4e15-b0b5-ad15217e72e3 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:12.688721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a9f57-f20a-4e15-b0b5-ad15217e72e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:12.706984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b73a9f57-f20a-4e15-b0b5-ad15217e72e3 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:12.708148 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b73a9f57-f20a-4e15-b0b5-ad15217e72e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:12.774289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 610/1221] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:01:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:01:13.043053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7afc0-bc3d-4612-ad5e-02d103238f1c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:13.144424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7afc0-bc3d-4612-ad5e-02d103238f1c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:13.179275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275a9a-4e83-4b31-9510-4ee478a3492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:13.179882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275a9a-4e83-4b31-9510-4ee478a3492e None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:13.180185 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275a9a-4e83-4b31-9510-4ee478a3492e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:13.180454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275a9a-4e83-4b31-9510-4ee478a3492e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:13.190638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc275a9a-4e83-4b31-9510-4ee478a3492e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-fc275a9a-4e83-4b31-9510-4ee478a3492e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:01:13.195286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:13.196405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:13.197442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 612/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 10:01:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:13.203506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a9f57-f20a-4e15-b0b5-ad15217e72e3 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:13.302668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a9f57-f20a-4e15-b0b5-ad15217e72e3 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:13.326182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:13.336865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35ffab-8716-4c1d-8dea-c28a8f47105b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:13.337567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ffab-8716-4c1d-8dea-c28a8f47105b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:13.337847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ffab-8716-4c1d-8dea-c28a8f47105b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:13.338118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ffab-8716-4c1d-8dea-c28a8f47105b None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:13.338776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35ffab-8716-4c1d-8dea-c28a8f47105b None admin] RBAC: Adding query filter params (name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:13.339076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35ffab-8716-4c1d-8dea-c28a8f47105b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:13.340434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35ffab-8716-4c1d-8dea-c28a8f47105b None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:13.348986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 611/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 10:01:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:13.354894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275a9a-4e83-4b31-9510-4ee478a3492e None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:13.399959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:13.457515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275a9a-4e83-4b31-9510-4ee478a3492e None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:13.493500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c832e4c-5554-4a57-9803-04122baccb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:13.494125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c832e4c-5554-4a57-9803-04122baccb56 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:13.494409 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c832e4c-5554-4a57-9803-04122baccb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:13.494846 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c832e4c-5554-4a57-9803-04122baccb56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:13.505593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c832e4c-5554-4a57-9803-04122baccb56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:13.507014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c832e4c-5554-4a57-9803-04122baccb56 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-4c832e4c-5554-4a57-9803-04122baccb56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:13.522023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:13.523188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:13.524174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:13.524174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:13.524174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:13.524174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 613/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:13.529765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35ffab-8716-4c1d-8dea-c28a8f47105b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:13.629192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35ffab-8716-4c1d-8dea-c28a8f47105b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:13.662878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19d15f-f69d-4e62-8a83-0f3c16dfcbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:13.663477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19d15f-f69d-4e62-8a83-0f3c16dfcbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:13.663755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19d15f-f69d-4e62-8a83-0f3c16dfcbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:13.664027 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19d15f-f69d-4e62-8a83-0f3c16dfcbe8 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:13.664664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19d15f-f69d-4e62-8a83-0f3c16dfcbe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:13.664994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19d15f-f69d-4e62-8a83-0f3c16dfcbe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:13.666351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19d15f-f69d-4e62-8a83-0f3c16dfcbe8 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:13.678045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 612/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:13.683911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c832e4c-5554-4a57-9803-04122baccb56 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:13.776578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:13.783403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c832e4c-5554-4a57-9803-04122baccb56 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:13.817759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbf7cc-7f41-44b9-82e5-9785160c39f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:13.818388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbf7cc-7f41-44b9-82e5-9785160c39f0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:13.818669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbf7cc-7f41-44b9-82e5-9785160c39f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:13.818943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbf7cc-7f41-44b9-82e5-9785160c39f0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:13.827596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbf7cc-7f41-44b9-82e5-9785160c39f0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:13.829063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbf7cc-7f41-44b9-82e5-9785160c39f0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-adbbf7cc-7f41-44b9-82e5-9785160c39f0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:01:13.840123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:13.841354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:13.842520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:13.842520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:13.842520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:01:13.842520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:01:13.842520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:01:13.842520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:01:13.842520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:01:13.842520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:01:13.842520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:13.842520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:13.842520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 614/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:13.847097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19d15f-f69d-4e62-8a83-0f3c16dfcbe8 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:13.895906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:13.948149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19d15f-f69d-4e62-8a83-0f3c16dfcbe8 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:13.982430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5632a4-ea0e-4f3f-9b80-10c259d98577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:13.983321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5632a4-ea0e-4f3f-9b80-10c259d98577 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:13.983600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5632a4-ea0e-4f3f-9b80-10c259d98577 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:13.983869 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5632a4-ea0e-4f3f-9b80-10c259d98577 None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:13.984596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5632a4-ea0e-4f3f-9b80-10c259d98577 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:13.984929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5632a4-ea0e-4f3f-9b80-10c259d98577 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:13.986372 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5632a4-ea0e-4f3f-9b80-10c259d98577 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:14.004068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 613/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Aug 24 10:01:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:01:14.009782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbf7cc-7f41-44b9-82e5-9785160c39f0 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:14.107704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbf7cc-7f41-44b9-82e5-9785160c39f0 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:14.140836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137255c-cb5b-4c5d-bf8b-67fcfa6fa9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:14.141474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137255c-cb5b-4c5d-bf8b-67fcfa6fa9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.141752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137255c-cb5b-4c5d-bf8b-67fcfa6fa9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.142023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137255c-cb5b-4c5d-bf8b-67fcfa6fa9b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.152422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137255c-cb5b-4c5d-bf8b-67fcfa6fa9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:14.153861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137255c-cb5b-4c5d-bf8b-67fcfa6fa9b2 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-b137255c-cb5b-4c5d-bf8b-67fcfa6fa9b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:14.167799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:14.174688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:14.175827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:14.175827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:14.175827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:14.175827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 615/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:14.176124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5632a4-ea0e-4f3f-9b80-10c259d98577 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:14.279358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5632a4-ea0e-4f3f-9b80-10c259d98577 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:14.281739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:14.314385 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2a61b-ed0e-4592-a820-cb19ab39643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:14.314986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2a61b-ed0e-4592-a820-cb19ab39643e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.315265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2a61b-ed0e-4592-a820-cb19ab39643e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.315567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2a61b-ed0e-4592-a820-cb19ab39643e None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.316204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2a61b-ed0e-4592-a820-cb19ab39643e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:14.316504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2a61b-ed0e-4592-a820-cb19ab39643e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:14.318173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2a61b-ed0e-4592-a820-cb19ab39643e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:14.329595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 614/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:14.335613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137255c-cb5b-4c5d-bf8b-67fcfa6fa9b2 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:14.345654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:14.439330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137255c-cb5b-4c5d-bf8b-67fcfa6fa9b2 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:14.472566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7408dd-16f0-4f9f-b93b-e3d761dfa3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:14.473210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7408dd-16f0-4f9f-b93b-e3d761dfa3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.473488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7408dd-16f0-4f9f-b93b-e3d761dfa3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.473758 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7408dd-16f0-4f9f-b93b-e3d761dfa3a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.480265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7408dd-16f0-4f9f-b93b-e3d761dfa3a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:14.481731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7408dd-16f0-4f9f-b93b-e3d761dfa3a7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-be7408dd-16f0-4f9f-b93b-e3d761dfa3a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:01:14.493365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:14.494425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:14.495496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:01:14.495496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:01:14.495496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:01:14.495496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:01:14.495496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:01:14.495496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:01:14.495496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:14.495496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:14.495496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 616/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 10:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:14.500377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2a61b-ed0e-4592-a820-cb19ab39643e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:14.598544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2a61b-ed0e-4592-a820-cb19ab39643e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:14.631461 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929d7c7-e4af-4707-a68a-c4cae7bc78b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:14.632063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929d7c7-e4af-4707-a68a-c4cae7bc78b1 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.632380 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929d7c7-e4af-4707-a68a-c4cae7bc78b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.632608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929d7c7-e4af-4707-a68a-c4cae7bc78b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.633375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929d7c7-e4af-4707-a68a-c4cae7bc78b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:14.633617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929d7c7-e4af-4707-a68a-c4cae7bc78b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:14.635034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929d7c7-e4af-4707-a68a-c4cae7bc78b1 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:14.647191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 615/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 10:01:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:14.654258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7408dd-16f0-4f9f-b93b-e3d761dfa3a7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:14.721734 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:14.754635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7408dd-16f0-4f9f-b93b-e3d761dfa3a7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:14.787990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367ccd1-f93f-4695-ad60-ba15e0fdf6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:14.788640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367ccd1-f93f-4695-ad60-ba15e0fdf6dc None admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.788948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367ccd1-f93f-4695-ad60-ba15e0fdf6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.789225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367ccd1-f93f-4695-ad60-ba15e0fdf6dc None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/b1aa47a883604336a8ab90d324470477/roles/f9446582c6a34010bd3ae841eba530bc` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.802910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367ccd1-f93f-4695-ad60-ba15e0fdf6dc None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=b1aa47a883604336a8ab90d324470477, role_id=f9446582c6a34010bd3ae841eba530bc)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:14.804383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367ccd1-f93f-4695-ad60-ba15e0fdf6dc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:14.908601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 617/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 10:01:14 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/b1aa47a883604336a8ab90d324470477/roles/f9446582c6a34010bd3ae841eba530bc => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:16.832367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d721e-5f38-4834-a35a-64fffa56bab9 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:16.832957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d721e-5f38-4834-a35a-64fffa56bab9 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:16.832957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d721e-5f38-4834-a35a-64fffa56bab9 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:16.834350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 616/1233] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:16.839543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e8052-c458-4dd8-800f-5f2955ba5147 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:16.839932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e8052-c458-4dd8-800f-5f2955ba5147 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:16.840094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e8052-c458-4dd8-800f-5f2955ba5147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:16.859355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-984e8052-c458-4dd8-800f-5f2955ba5147 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:16.860554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-984e8052-c458-4dd8-800f-5f2955ba5147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:16.927030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 618/1234] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:01:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:01:17.197470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d721e-5f38-4834-a35a-64fffa56bab9 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:17.223319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:17.298897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d721e-5f38-4834-a35a-64fffa56bab9 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:17.332863 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de075c-f7e3-43a7-bf2c-36491ca58b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:17.333516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de075c-f7e3-43a7-bf2c-36491ca58b2e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:17.333795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de075c-f7e3-43a7-bf2c-36491ca58b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:17.334067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de075c-f7e3-43a7-bf2c-36491ca58b2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:17.343924 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de075c-f7e3-43a7-bf2c-36491ca58b2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b0de075c-f7e3-43a7-bf2c-36491ca58b2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:01:17.348837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:17.350240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:17.351292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 617/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Aug 24 10:01:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:17.356985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e8052-c458-4dd8-800f-5f2955ba5147 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:17.404220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:17.455765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e8052-c458-4dd8-800f-5f2955ba5147 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:17.489057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1508d99-4945-49ca-9a20-a71d67005d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:17.489658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1508d99-4945-49ca-9a20-a71d67005d04 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:17.489938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1508d99-4945-49ca-9a20-a71d67005d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:17.490214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1508d99-4945-49ca-9a20-a71d67005d04 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:17.490908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1508d99-4945-49ca-9a20-a71d67005d04 None admin] RBAC: Adding query filter params (name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:17.491253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1508d99-4945-49ca-9a20-a71d67005d04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:17.492619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1508d99-4945-49ca-9a20-a71d67005d04 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:17.501270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 619/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Aug 24 10:01:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:17.507107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de075c-f7e3-43a7-bf2c-36491ca58b2e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:17.592088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:17.607604 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de075c-f7e3-43a7-bf2c-36491ca58b2e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:17.641304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361ecb6-45cf-4b4f-8ff7-eba1d78d267c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:17.641944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361ecb6-45cf-4b4f-8ff7-eba1d78d267c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:17.642206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361ecb6-45cf-4b4f-8ff7-eba1d78d267c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:17.642479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361ecb6-45cf-4b4f-8ff7-eba1d78d267c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:17.652518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361ecb6-45cf-4b4f-8ff7-eba1d78d267c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:17.654165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361ecb6-45cf-4b4f-8ff7-eba1d78d267c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-7361ecb6-45cf-4b4f-8ff7-eba1d78d267c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:17.668379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:17.670137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:17.671231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:17.671231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:17.671231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:17.671231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 618/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:17.676062 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1508d99-4945-49ca-9a20-a71d67005d04 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:17.777219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1508d99-4945-49ca-9a20-a71d67005d04 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:17.786038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:17.811072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951760-a760-4ab1-899b-0f632d812b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:17.811674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951760-a760-4ab1-899b-0f632d812b6f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:17.811959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951760-a760-4ab1-899b-0f632d812b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:17.812233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951760-a760-4ab1-899b-0f632d812b6f None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:17.812897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7951760-a760-4ab1-899b-0f632d812b6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:17.813208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7951760-a760-4ab1-899b-0f632d812b6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:17.814587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7951760-a760-4ab1-899b-0f632d812b6f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:17.826695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 620/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:01:17.832585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361ecb6-45cf-4b4f-8ff7-eba1d78d267c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:17.934636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361ecb6-45cf-4b4f-8ff7-eba1d78d267c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:17.970833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86221f06-9788-4e51-8a56-e046295f6b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:17.971464 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86221f06-9788-4e51-8a56-e046295f6b78 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:17.971742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86221f06-9788-4e51-8a56-e046295f6b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:17.972012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86221f06-9788-4e51-8a56-e046295f6b78 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:17.974794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:17.981168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86221f06-9788-4e51-8a56-e046295f6b78 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:17.982719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86221f06-9788-4e51-8a56-e046295f6b78 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-86221f06-9788-4e51-8a56-e046295f6b78 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:01:17.993639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:17.995457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:17.998121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:17.998121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:17.998121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:01:17.998121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:01:17.998121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:01:17.998121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:01:17.998121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:01:17.998121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:01:17.998121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:17.998121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:17.998121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 619/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:18.002752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951760-a760-4ab1-899b-0f632d812b6f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:18.101834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951760-a760-4ab1-899b-0f632d812b6f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:18.135728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec760d4-0ee0-470a-8fdd-a604677ef40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:18.136352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec760d4-0ee0-470a-8fdd-a604677ef40b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.136637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec760d4-0ee0-470a-8fdd-a604677ef40b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.136937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec760d4-0ee0-470a-8fdd-a604677ef40b None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.137663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec760d4-0ee0-470a-8fdd-a604677ef40b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:18.138587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec760d4-0ee0-470a-8fdd-a604677ef40b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:18.140012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec760d4-0ee0-470a-8fdd-a604677ef40b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:18.157968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 621/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Aug 24 10:01:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:18.163780 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86221f06-9788-4e51-8a56-e046295f6b78 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:18.267725 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86221f06-9788-4e51-8a56-e046295f6b78 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:18.301354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4828611-ce45-41d4-a316-b39ed5c5206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:18.301960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4828611-ce45-41d4-a316-b39ed5c5206d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.302266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4828611-ce45-41d4-a316-b39ed5c5206d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.302540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4828611-ce45-41d4-a316-b39ed5c5206d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.363163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:18.369600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4828611-ce45-41d4-a316-b39ed5c5206d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:18.371045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4828611-ce45-41d4-a316-b39ed5c5206d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:18.469038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 620/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 10:01:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:01:18.474909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec760d4-0ee0-470a-8fdd-a604677ef40b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:18.506188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:18.579442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec760d4-0ee0-470a-8fdd-a604677ef40b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:18.614375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911cf32-1025-4c24-9ed1-8ce2b22b27c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:18.614934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911cf32-1025-4c24-9ed1-8ce2b22b27c6 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.615214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911cf32-1025-4c24-9ed1-8ce2b22b27c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.615488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911cf32-1025-4c24-9ed1-8ce2b22b27c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.625902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911cf32-1025-4c24-9ed1-8ce2b22b27c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:18.627457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911cf32-1025-4c24-9ed1-8ce2b22b27c6 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-d911cf32-1025-4c24-9ed1-8ce2b22b27c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:18.641636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:18.657161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:18.658325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:18.658325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:18.658325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:18.658325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 622/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:18.658645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4828611-ce45-41d4-a316-b39ed5c5206d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:18.741411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:18.747056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4828611-ce45-41d4-a316-b39ed5c5206d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:18.781644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657d2cb-5a1a-43e4-ad61-1de7034be12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:18.782276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657d2cb-5a1a-43e4-ad61-1de7034be12a None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.782553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657d2cb-5a1a-43e4-ad61-1de7034be12a None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.782829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657d2cb-5a1a-43e4-ad61-1de7034be12a None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.783466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657d2cb-5a1a-43e4-ad61-1de7034be12a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:18.783776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657d2cb-5a1a-43e4-ad61-1de7034be12a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:18.785177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657d2cb-5a1a-43e4-ad61-1de7034be12a None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:18.796727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 621/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:18.802574 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911cf32-1025-4c24-9ed1-8ce2b22b27c6 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:18.885360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:18.906669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911cf32-1025-4c24-9ed1-8ce2b22b27c6 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:18.939530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2750de4-c7a9-4c19-8081-a9d56bec5b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:18.940137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2750de4-c7a9-4c19-8081-a9d56bec5b30 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.940414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2750de4-c7a9-4c19-8081-a9d56bec5b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.940684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2750de4-c7a9-4c19-8081-a9d56bec5b30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.947136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2750de4-c7a9-4c19-8081-a9d56bec5b30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:18.948555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2750de4-c7a9-4c19-8081-a9d56bec5b30 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-c2750de4-c7a9-4c19-8081-a9d56bec5b30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:01:18.959640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:18.960712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:18.961786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:01:18.961786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:01:18.961786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:01:18.961786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:01:18.961786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:01:18.961786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:01:18.961786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:18.961786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:18.961786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 623/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 10:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:18.967469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657d2cb-5a1a-43e4-ad61-1de7034be12a None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:19.067793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657d2cb-5a1a-43e4-ad61-1de7034be12a None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:19.101435 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dca349-e7a9-4aff-9040-3b9ed78bf819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:19.102032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dca349-e7a9-4aff-9040-3b9ed78bf819 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.102311 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dca349-e7a9-4aff-9040-3b9ed78bf819 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.102640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dca349-e7a9-4aff-9040-3b9ed78bf819 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.103325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dca349-e7a9-4aff-9040-3b9ed78bf819 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:19.103632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dca349-e7a9-4aff-9040-3b9ed78bf819 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:19.105081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dca349-e7a9-4aff-9040-3b9ed78bf819 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:19.117508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 622/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 10:01:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:19.123907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2750de4-c7a9-4c19-8081-a9d56bec5b30 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:19.222643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2750de4-c7a9-4c19-8081-a9d56bec5b30 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:19.254558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4648a0-6cf0-4ff8-af16-31a4e957b6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:19.255149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4648a0-6cf0-4ff8-af16-31a4e957b6fc None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.255427 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4648a0-6cf0-4ff8-af16-31a4e957b6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.255697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4648a0-6cf0-4ff8-af16-31a4e957b6fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.256417 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4648a0-6cf0-4ff8-af16-31a4e957b6fc None admin] RBAC: Adding query filter params (role.id=f9446582c6a34010bd3ae841eba530bc, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=b1aa47a883604336a8ab90d324470477) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:19.256723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4648a0-6cf0-4ff8-af16-31a4e957b6fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:19.258166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4648a0-6cf0-4ff8-af16-31a4e957b6fc None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:19.264461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 624/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 10:01:19 2022] GET /identity/v3/role_assignments?user.id=b1aa47a883604336a8ab90d324470477&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=f9446582c6a34010bd3ae841eba530bc => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:21.240448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246d810-d071-4cbe-9ece-b33eb0d8820e None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:21.241034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246d810-d071-4cbe-9ece-b33eb0d8820e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:21.241034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246d810-d071-4cbe-9ece-b33eb0d8820e None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:21.242441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 623/1247] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:21.247909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2703a3-2363-44a1-8f3b-1187f04406fa None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:21.248321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2703a3-2363-44a1-8f3b-1187f04406fa None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:21.251335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2703a3-2363-44a1-8f3b-1187f04406fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:21.254828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:21.267107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-7e2703a3-2363-44a1-8f3b-1187f04406fa None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:21.268302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7e2703a3-2363-44a1-8f3b-1187f04406fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:21.335941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 625/1248] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:01:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 10:01:21.619679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246d810-d071-4cbe-9ece-b33eb0d8820e None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:21.638738 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:21.722490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246d810-d071-4cbe-9ece-b33eb0d8820e None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:21.757268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b6d70-bc1b-46b6-83c9-d393cb9ea29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:21.757866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b6d70-bc1b-46b6-83c9-d393cb9ea29d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:21.758143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b6d70-bc1b-46b6-83c9-d393cb9ea29d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:21.758414 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b6d70-bc1b-46b6-83c9-d393cb9ea29d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:21.768515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b6d70-bc1b-46b6-83c9-d393cb9ea29d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b95b6d70-bc1b-46b6-83c9-d393cb9ea29d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:01:21.773186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:21.777053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:21.778132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 624/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 10:01:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:21.781592 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2703a3-2363-44a1-8f3b-1187f04406fa None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:21.879243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2703a3-2363-44a1-8f3b-1187f04406fa None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:21.910386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:21.915449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850cd7b-7d82-4e91-9edd-7e2fb1531cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:21.916061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850cd7b-7d82-4e91-9edd-7e2fb1531cae None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:21.916347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850cd7b-7d82-4e91-9edd-7e2fb1531cae None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:21.916622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850cd7b-7d82-4e91-9edd-7e2fb1531cae None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:21.917376 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850cd7b-7d82-4e91-9edd-7e2fb1531cae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:21.917680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850cd7b-7d82-4e91-9edd-7e2fb1531cae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:21.919684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850cd7b-7d82-4e91-9edd-7e2fb1531cae None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:21.928081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 626/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 10:01:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:21.934107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b6d70-bc1b-46b6-83c9-d393cb9ea29d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.010723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:22.033316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b6d70-bc1b-46b6-83c9-d393cb9ea29d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.067689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5d0bc-18ea-4ddf-b5e9-afd17f655ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.068333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5d0bc-18ea-4ddf-b5e9-afd17f655ffa None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.068604 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5d0bc-18ea-4ddf-b5e9-afd17f655ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.068904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5d0bc-18ea-4ddf-b5e9-afd17f655ffa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.079964 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5d0bc-18ea-4ddf-b5e9-afd17f655ffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.081446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5d0bc-18ea-4ddf-b5e9-afd17f655ffa None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-c7f5d0bc-18ea-4ddf-b5e9-afd17f655ffa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:22.096630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:22.098277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:22.099409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:22.099409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:22.099409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:22.099409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 625/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:22.103930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850cd7b-7d82-4e91-9edd-7e2fb1531cae None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.203444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850cd7b-7d82-4e91-9edd-7e2fb1531cae None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.236401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6c860-04f3-4bc2-8066-5a3941424cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.237023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6c860-04f3-4bc2-8066-5a3941424cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.237303 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6c860-04f3-4bc2-8066-5a3941424cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.237576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6c860-04f3-4bc2-8066-5a3941424cb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.238215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6c860-04f3-4bc2-8066-5a3941424cb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:22.238515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6c860-04f3-4bc2-8066-5a3941424cb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.239874 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6c860-04f3-4bc2-8066-5a3941424cb4 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.251416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 627/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:22.257443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5d0bc-18ea-4ddf-b5e9-afd17f655ffa None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.361268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5d0bc-18ea-4ddf-b5e9-afd17f655ffa None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.387193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:22.395953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55458aeb-7da9-44f8-bb44-e914c4e62655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.396587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55458aeb-7da9-44f8-bb44-e914c4e62655 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.396881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55458aeb-7da9-44f8-bb44-e914c4e62655 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.397179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55458aeb-7da9-44f8-bb44-e914c4e62655 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.406086 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55458aeb-7da9-44f8-bb44-e914c4e62655 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.407529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55458aeb-7da9-44f8-bb44-e914c4e62655 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-55458aeb-7da9-44f8-bb44-e914c4e62655 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:01:22.418860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:22.425502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:22.426676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:22.426676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:22.426676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:01:22.426676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:01:22.426676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:01:22.426676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:01:22.426676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:01:22.426676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:01:22.426676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:22.426676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:22.426676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 626/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:22.427267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6c860-04f3-4bc2-8066-5a3941424cb4 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.483443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:22.527427 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6c860-04f3-4bc2-8066-5a3941424cb4 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.561293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae305f03-a959-44b5-be86-c71fb3a3b492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.561892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae305f03-a959-44b5-be86-c71fb3a3b492 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.562170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae305f03-a959-44b5-be86-c71fb3a3b492 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.562441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae305f03-a959-44b5-be86-c71fb3a3b492 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.563162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae305f03-a959-44b5-be86-c71fb3a3b492 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:22.563463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae305f03-a959-44b5-be86-c71fb3a3b492 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.564891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae305f03-a959-44b5-be86-c71fb3a3b492 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.582846 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 628/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Aug 24 10:01:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:22.588938 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55458aeb-7da9-44f8-bb44-e914c4e62655 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.688918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55458aeb-7da9-44f8-bb44-e914c4e62655 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.722660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7470c54-3aad-472a-8e31-62d7cef71afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.723253 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7470c54-3aad-472a-8e31-62d7cef71afc None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.723530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7470c54-3aad-472a-8e31-62d7cef71afc None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.723801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7470c54-3aad-472a-8e31-62d7cef71afc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.777966 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:22.789592 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7470c54-3aad-472a-8e31-62d7cef71afc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.793304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7470c54-3aad-472a-8e31-62d7cef71afc None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.889956 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 627/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 10:01:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:01:22.895915 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae305f03-a959-44b5-be86-c71fb3a3b492 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.001159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae305f03-a959-44b5-be86-c71fb3a3b492 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.014047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:23.034827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae8697-89a1-42b0-ba1c-e7bfa682d461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.035383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae8697-89a1-42b0-ba1c-e7bfa682d461 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.035665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae8697-89a1-42b0-ba1c-e7bfa682d461 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.035938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae8697-89a1-42b0-ba1c-e7bfa682d461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.046627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae8697-89a1-42b0-ba1c-e7bfa682d461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.048061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae8697-89a1-42b0-ba1c-e7bfa682d461 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-ecae8697-89a1-42b0-ba1c-e7bfa682d461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:23.063405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:23.064512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:23.065534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:23.065534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:23.065534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:23.065534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 629/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:23.070492 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7470c54-3aad-472a-8e31-62d7cef71afc None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.157746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:23.169973 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7470c54-3aad-472a-8e31-62d7cef71afc None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.203405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ff65e-2d04-4fc3-9f0b-3ec684eef61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.204003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ff65e-2d04-4fc3-9f0b-3ec684eef61e None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.204283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ff65e-2d04-4fc3-9f0b-3ec684eef61e None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.204559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ff65e-2d04-4fc3-9f0b-3ec684eef61e None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.205227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ff65e-2d04-4fc3-9f0b-3ec684eef61e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:23.205535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ff65e-2d04-4fc3-9f0b-3ec684eef61e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.206895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ff65e-2d04-4fc3-9f0b-3ec684eef61e None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.218069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 628/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:23.223860 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae8697-89a1-42b0-ba1c-e7bfa682d461 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.327537 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae8697-89a1-42b0-ba1c-e7bfa682d461 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.361810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715320f-4f6b-41e4-89ea-4aaec06cd4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.362425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715320f-4f6b-41e4-89ea-4aaec06cd4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.362710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715320f-4f6b-41e4-89ea-4aaec06cd4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.362975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715320f-4f6b-41e4-89ea-4aaec06cd4c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.369448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715320f-4f6b-41e4-89ea-4aaec06cd4c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.374189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715320f-4f6b-41e4-89ea-4aaec06cd4c0 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-1715320f-4f6b-41e4-89ea-4aaec06cd4c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:01:23.383099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:23.384038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:23.385078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:01:23.385078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:01:23.385078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:01:23.385078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:01:23.385078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:01:23.385078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:01:23.385078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:23.385078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:23.385078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 630/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 10:01:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:23.390497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ff65e-2d04-4fc3-9f0b-3ec684eef61e None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.491071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ff65e-2d04-4fc3-9f0b-3ec684eef61e None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.523508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1675d-ae1b-4989-aec7-c9c54f07ece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.524106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1675d-ae1b-4989-aec7-c9c54f07ece3 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.524383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1675d-ae1b-4989-aec7-c9c54f07ece3 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.524653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1675d-ae1b-4989-aec7-c9c54f07ece3 None admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.525375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1675d-ae1b-4989-aec7-c9c54f07ece3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:23.525679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1675d-ae1b-4989-aec7-c9c54f07ece3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.527369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1675d-ae1b-4989-aec7-c9c54f07ece3 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.539515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 629/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 10:01:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:23.546614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715320f-4f6b-41e4-89ea-4aaec06cd4c0 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.572708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:23.647727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715320f-4f6b-41e4-89ea-4aaec06cd4c0 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.681570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf64890-349a-4f4c-a0c3-ad486d119374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.682165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf64890-349a-4f4c-a0c3-ad486d119374 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.682446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf64890-349a-4f4c-a0c3-ad486d119374 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.682720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf64890-349a-4f4c-a0c3-ad486d119374 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.683441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf64890-349a-4f4c-a0c3-ad486d119374 None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=b1aa47a883604336a8ab90d324470477) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:23.683746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf64890-349a-4f4c-a0c3-ad486d119374 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.685360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf64890-349a-4f4c-a0c3-ad486d119374 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.690643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 631/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 10:01:23 2022] GET /identity/v3/role_assignments?user.id=b1aa47a883604336a8ab90d324470477&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:25.591280 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546963-c09e-4141-9a85-0e11baa16a46 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:25.591785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546963-c09e-4141-9a85-0e11baa16a46 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:25.591785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546963-c09e-4141-9a85-0e11baa16a46 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:25.593140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 630/1261] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:25.598373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061822a2-9449-4f0a-a0d6-6545aa459277 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:25.598753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061822a2-9449-4f0a-a0d6-6545aa459277 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:25.598927 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061822a2-9449-4f0a-a0d6-6545aa459277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:25.617727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-061822a2-9449-4f0a-a0d6-6545aa459277 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:25.619581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-061822a2-9449-4f0a-a0d6-6545aa459277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:25.697460 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 632/1262] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:01:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:01:25.967231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12546963-c09e-4141-9a85-0e11baa16a46 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:25.978038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:26.068959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12546963-c09e-4141-9a85-0e11baa16a46 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.102357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6ab9d-346f-4a05-9d4d-1399d137fd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.102973 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6ab9d-346f-4a05-9d4d-1399d137fd3b None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.103290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6ab9d-346f-4a05-9d4d-1399d137fd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.103564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6ab9d-346f-4a05-9d4d-1399d137fd3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.114408 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6ab9d-346f-4a05-9d4d-1399d137fd3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-b4d6ab9d-346f-4a05-9d4d-1399d137fd3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:01:26.120022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:26.121160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:26.122866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 631/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 10:01:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:26.126509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061822a2-9449-4f0a-a0d6-6545aa459277 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:26.144798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:26.224625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061822a2-9449-4f0a-a0d6-6545aa459277 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.259089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ba3d3-e380-48ac-802d-59120fe3ea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.259690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ba3d3-e380-48ac-802d-59120fe3ea90 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.259969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ba3d3-e380-48ac-802d-59120fe3ea90 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.260241 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ba3d3-e380-48ac-802d-59120fe3ea90 None admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.260923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ba3d3-e380-48ac-802d-59120fe3ea90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:26.261228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ba3d3-e380-48ac-802d-59120fe3ea90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.262624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ba3d3-e380-48ac-802d-59120fe3ea90 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.271234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 633/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 10:01:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:26.277176 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6ab9d-346f-4a05-9d4d-1399d137fd3b None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:26.345969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:26.381087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6ab9d-346f-4a05-9d4d-1399d137fd3b None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.415003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea1840-d258-4fe4-b1a7-7bf93a02fdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.415664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea1840-d258-4fe4-b1a7-7bf93a02fdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.415945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea1840-d258-4fe4-b1a7-7bf93a02fdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.416220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea1840-d258-4fe4-b1a7-7bf93a02fdc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.427202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea1840-d258-4fe4-b1a7-7bf93a02fdc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.428649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea1840-d258-4fe4-b1a7-7bf93a02fdc0 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-64ea1840-d258-4fe4-b1a7-7bf93a02fdc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:26.442878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:26.444236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:26.445592 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:26.445592 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:26.445592 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:26.445592 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 632/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:26.449846 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ba3d3-e380-48ac-802d-59120fe3ea90 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:26.533790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:26.553261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ba3d3-e380-48ac-802d-59120fe3ea90 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.585369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3261bf-b45e-43cf-97e3-6c8fa0491d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.585966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3261bf-b45e-43cf-97e3-6c8fa0491d59 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.586249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3261bf-b45e-43cf-97e3-6c8fa0491d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.586526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3261bf-b45e-43cf-97e3-6c8fa0491d59 None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.587157 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3261bf-b45e-43cf-97e3-6c8fa0491d59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:26.587459 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3261bf-b45e-43cf-97e3-6c8fa0491d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.588898 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3261bf-b45e-43cf-97e3-6c8fa0491d59 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.600747 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 634/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:26.607170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea1840-d258-4fe4-b1a7-7bf93a02fdc0 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:26.708776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea1840-d258-4fe4-b1a7-7bf93a02fdc0 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.728600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:26.743338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a8f31-f587-4e44-a153-0cb90549981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.744331 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a8f31-f587-4e44-a153-0cb90549981c None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.744785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a8f31-f587-4e44-a153-0cb90549981c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.745261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a8f31-f587-4e44-a153-0cb90549981c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.754300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a8f31-f587-4e44-a153-0cb90549981c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.755738 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a8f31-f587-4e44-a153-0cb90549981c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-662a8f31-f587-4e44-a153-0cb90549981c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:01:26.767042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:26.768154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:26.769161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:26.769161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:26.769161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:01:26.769161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:01:26.769161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:01:26.769161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:01:26.769161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:01:26.769161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:01:26.769161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:26.769161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:26.769161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 633/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:26.774004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3261bf-b45e-43cf-97e3-6c8fa0491d59 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:26.875365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3261bf-b45e-43cf-97e3-6c8fa0491d59 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.909647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0601c05-bc79-449a-90d5-6ed52378d9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.910248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0601c05-bc79-449a-90d5-6ed52378d9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.910527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0601c05-bc79-449a-90d5-6ed52378d9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.910797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0601c05-bc79-449a-90d5-6ed52378d9c7 None admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.911516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0601c05-bc79-449a-90d5-6ed52378d9c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:26.911818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0601c05-bc79-449a-90d5-6ed52378d9c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.913264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0601c05-bc79-449a-90d5-6ed52378d9c7 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.922821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:26.932000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 635/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Aug 24 10:01:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:26.938794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a8f31-f587-4e44-a153-0cb90549981c None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.039209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a8f31-f587-4e44-a153-0cb90549981c None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.076136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6a2b7-e7ce-4262-ac84-da5e87a3feb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.076766 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6a2b7-e7ce-4262-ac84-da5e87a3feb4 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.077072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6a2b7-e7ce-4262-ac84-da5e87a3feb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.077352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6a2b7-e7ce-4262-ac84-da5e87a3feb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.087677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6a2b7-e7ce-4262-ac84-da5e87a3feb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.089147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6a2b7-e7ce-4262-ac84-da5e87a3feb4 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-cdf6a2b7-e7ce-4262-ac84-da5e87a3feb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:27.105225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:27.107062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:27.108174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:27.108174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:27.108174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:27.108174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 634/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:27.113482 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0601c05-bc79-449a-90d5-6ed52378d9c7 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.211755 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0601c05-bc79-449a-90d5-6ed52378d9c7 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.246280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ca9b6-27dd-4be4-9ef8-0fcf4880693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.246876 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ca9b6-27dd-4be4-9ef8-0fcf4880693b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.247156 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ca9b6-27dd-4be4-9ef8-0fcf4880693b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.247473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ca9b6-27dd-4be4-9ef8-0fcf4880693b None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.248221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ca9b6-27dd-4be4-9ef8-0fcf4880693b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:27.248524 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ca9b6-27dd-4be4-9ef8-0fcf4880693b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.249917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ca9b6-27dd-4be4-9ef8-0fcf4880693b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.261213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 636/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:27.266927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6a2b7-e7ce-4262-ac84-da5e87a3feb4 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.300899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:27.371470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6a2b7-e7ce-4262-ac84-da5e87a3feb4 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.406732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8eaf62-c7b2-4004-b175-9b8aecbbfa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.407356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eaf62-c7b2-4004-b175-9b8aecbbfa76 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.407635 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eaf62-c7b2-4004-b175-9b8aecbbfa76 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.407911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eaf62-c7b2-4004-b175-9b8aecbbfa76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.414544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8eaf62-c7b2-4004-b175-9b8aecbbfa76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.415983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8eaf62-c7b2-4004-b175-9b8aecbbfa76 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-4b8eaf62-c7b2-4004-b175-9b8aecbbfa76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:01:27.427590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:27.429151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:27.430302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:01:27.430302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:01:27.430302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:01:27.430302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:01:27.430302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:01:27.430302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:01:27.430302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:27.430302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:27.430302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 635/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 10:01:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:27.434703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ca9b6-27dd-4be4-9ef8-0fcf4880693b None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.492457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:27.533529 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ca9b6-27dd-4be4-9ef8-0fcf4880693b None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.565503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f67fa-0db7-430a-b11f-dbdb06d4183b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.566110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f67fa-0db7-430a-b11f-dbdb06d4183b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.566454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f67fa-0db7-430a-b11f-dbdb06d4183b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.566732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f67fa-0db7-430a-b11f-dbdb06d4183b None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.567408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f67fa-0db7-430a-b11f-dbdb06d4183b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:27.567712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f67fa-0db7-430a-b11f-dbdb06d4183b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.569166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f67fa-0db7-430a-b11f-dbdb06d4183b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.581820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 637/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 10:01:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:27.587848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8eaf62-c7b2-4004-b175-9b8aecbbfa76 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.678479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:27.687476 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8eaf62-c7b2-4004-b175-9b8aecbbfa76 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.721192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7f4f2-d426-41c1-aef2-2cd6949d08e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.721840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7f4f2-d426-41c1-aef2-2cd6949d08e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.722118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7f4f2-d426-41c1-aef2-2cd6949d08e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.722388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7f4f2-d426-41c1-aef2-2cd6949d08e1 None admin] PATH_INFO: `/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/b1aa47a883604336a8ab90d324470477/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.724254 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7f4f2-d426-41c1-aef2-2cd6949d08e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=80e7b34d5e4a480691bed8235d4722c8, user_id=b1aa47a883604336a8ab90d324470477, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.725793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7f4f2-d426-41c1-aef2-2cd6949d08e1 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.789936 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 636/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Aug 24 10:01:27 2022] PUT /identity/v3/projects/80e7b34d5e4a480691bed8235d4722c8/users/b1aa47a883604336a8ab90d324470477/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:29.696360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20769831-12c5-4aea-af71-e57ccb2b88bd None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:29.734365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20769831-12c5-4aea-af71-e57ccb2b88bd None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:29.734365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20769831-12c5-4aea-af71-e57ccb2b88bd None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:29.734365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 638/1274] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:29.734666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b52aa-7f31-4294-ba6f-011400f82286 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:29.734666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b52aa-7f31-4294-ba6f-011400f82286 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:29.734666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b52aa-7f31-4294-ba6f-011400f82286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:29.734666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-2a4b52aa-7f31-4294-ba6f-011400f82286 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:29.734666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2a4b52aa-7f31-4294-ba6f-011400f82286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:29.792419 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 637/1275] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:01:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:01:30.067604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20769831-12c5-4aea-af71-e57ccb2b88bd None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:30.169063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20769831-12c5-4aea-af71-e57ccb2b88bd None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:30.186607 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:30.205791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcffaf8-054f-48c2-91fd-8c48a0488741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:30.206498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcffaf8-054f-48c2-91fd-8c48a0488741 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:30.206759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcffaf8-054f-48c2-91fd-8c48a0488741 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:30.207031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcffaf8-054f-48c2-91fd-8c48a0488741 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:30.216944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcffaf8-054f-48c2-91fd-8c48a0488741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-cdcffaf8-054f-48c2-91fd-8c48a0488741 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 10:01:30.221542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:30.222816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:30.223963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 639/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Aug 24 10:01:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:30.228954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b52aa-7f31-4294-ba6f-011400f82286 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:30.315877 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:30.339732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b52aa-7f31-4294-ba6f-011400f82286 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:30.369358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c4834-4830-4aad-9f6b-646ab263d9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:30.369999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4834-4830-4aad-9f6b-646ab263d9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:30.370504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4834-4830-4aad-9f6b-646ab263d9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:30.370849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4834-4830-4aad-9f6b-646ab263d9f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:30.371520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c4834-4830-4aad-9f6b-646ab263d9f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:30.371828 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c4834-4830-4aad-9f6b-646ab263d9f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:30.373985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c4834-4830-4aad-9f6b-646ab263d9f5 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:30.382206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 638/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Aug 24 10:01:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:30.389172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcffaf8-054f-48c2-91fd-8c48a0488741 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:30.490660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcffaf8-054f-48c2-91fd-8c48a0488741 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:30.527065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e222a-6c0a-46e1-8699-9d93d5483f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:30.527682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e222a-6c0a-46e1-8699-9d93d5483f75 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:30.527959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e222a-6c0a-46e1-8699-9d93d5483f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:30.528255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e222a-6c0a-46e1-8699-9d93d5483f75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:30.538188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e222a-6c0a-46e1-8699-9d93d5483f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:30.539617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e222a-6c0a-46e1-8699-9d93d5483f75 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-f47e222a-6c0a-46e1-8699-9d93d5483f75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:30.554312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:30.555304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:30.556474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:30.556474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:30.556474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:30.556474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 640/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:30.562127 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c4834-4830-4aad-9f6b-646ab263d9f5 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:30.662755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c4834-4830-4aad-9f6b-646ab263d9f5 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:30.697889 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163fe82-bb7b-4262-b023-cc6beb73e3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:30.698525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163fe82-bb7b-4262-b023-cc6beb73e3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:30.701334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163fe82-bb7b-4262-b023-cc6beb73e3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:30.701334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163fe82-bb7b-4262-b023-cc6beb73e3f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:30.701334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163fe82-bb7b-4262-b023-cc6beb73e3f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:30.701334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163fe82-bb7b-4262-b023-cc6beb73e3f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:30.701816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163fe82-bb7b-4262-b023-cc6beb73e3f0 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:30.713370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 639/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:30.719211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e222a-6c0a-46e1-8699-9d93d5483f75 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:30.754426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:30.822771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e222a-6c0a-46e1-8699-9d93d5483f75 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:30.856209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e045d4-eb6f-446f-ba28-63c0f31d0411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:30.857466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e045d4-eb6f-446f-ba28-63c0f31d0411 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:30.857813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e045d4-eb6f-446f-ba28-63c0f31d0411 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:30.858078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e045d4-eb6f-446f-ba28-63c0f31d0411 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:30.865459 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e045d4-eb6f-446f-ba28-63c0f31d0411 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:30.866878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e045d4-eb6f-446f-ba28-63c0f31d0411 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-70e045d4-eb6f-446f-ba28-63c0f31d0411 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 10:01:30.877948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:30.879195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:30.880181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:30.880181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:30.880181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 10:01:30.880181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 10:01:30.880181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 10:01:30.880181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 10:01:30.880181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 10:01:30.880181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 10:01:30.880181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 10:01:30.880181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:30.880181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 641/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:30.885385 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163fe82-bb7b-4262-b023-cc6beb73e3f0 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:30.889571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:30.987629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163fe82-bb7b-4262-b023-cc6beb73e3f0 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:31.021148 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d0f80-57ce-482a-85d9-46286b54e08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:31.021740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d0f80-57ce-482a-85d9-46286b54e08d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:31.022016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d0f80-57ce-482a-85d9-46286b54e08d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:31.022284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d0f80-57ce-482a-85d9-46286b54e08d None admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:31.022995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d0f80-57ce-482a-85d9-46286b54e08d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:31.023292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d0f80-57ce-482a-85d9-46286b54e08d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:31.024695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d0f80-57ce-482a-85d9-46286b54e08d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:31.042180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 640/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Aug 24 10:01:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:31.049802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e045d4-eb6f-446f-ba28-63c0f31d0411 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:31.145727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:31.156446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e045d4-eb6f-446f-ba28-63c0f31d0411 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:31.191170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa659cb-815a-4dca-92e8-869930c64886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:31.191762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa659cb-815a-4dca-92e8-869930c64886 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:31.192038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa659cb-815a-4dca-92e8-869930c64886 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:31.192307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa659cb-815a-4dca-92e8-869930c64886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:31.256653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa659cb-815a-4dca-92e8-869930c64886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:31.258043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa659cb-815a-4dca-92e8-869930c64886 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:31.347089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:31.357636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 642/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Aug 24 10:01:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 10:01:31.363808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d0f80-57ce-482a-85d9-46286b54e08d None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:31.428419 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:31.465587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d0f80-57ce-482a-85d9-46286b54e08d None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:31.498835 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38499ad2-bf1e-4eb5-b23e-c70835aae573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:31.500724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38499ad2-bf1e-4eb5-b23e-c70835aae573 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:31.500724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38499ad2-bf1e-4eb5-b23e-c70835aae573 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:31.500724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38499ad2-bf1e-4eb5-b23e-c70835aae573 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:31.510236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38499ad2-bf1e-4eb5-b23e-c70835aae573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:31.511600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38499ad2-bf1e-4eb5-b23e-c70835aae573 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-38499ad2-bf1e-4eb5-b23e-c70835aae573 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 10:01:31.526487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:31.527907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 10:01:31.529801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 10:01:31.529801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 10:01:31.529801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:31.529801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 641/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:31.534253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa659cb-815a-4dca-92e8-869930c64886 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:31.634384 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa659cb-815a-4dca-92e8-869930c64886 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:31.668771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1610964-a1e6-44c1-8d3e-5aa5d8e0809a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:31.670039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1610964-a1e6-44c1-8d3e-5aa5d8e0809a None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:31.670384 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1610964-a1e6-44c1-8d3e-5aa5d8e0809a None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:31.670657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1610964-a1e6-44c1-8d3e-5aa5d8e0809a None admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:31.671299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1610964-a1e6-44c1-8d3e-5aa5d8e0809a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:31.671604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1610964-a1e6-44c1-8d3e-5aa5d8e0809a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:31.673056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1610964-a1e6-44c1-8d3e-5aa5d8e0809a None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:31.684305 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 643/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Aug 24 10:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:31.690239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38499ad2-bf1e-4eb5-b23e-c70835aae573 None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:31.792945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38499ad2-bf1e-4eb5-b23e-c70835aae573 None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:31.807097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:31.826824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9173e-c345-451c-b5da-50c1b27a3fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:31.828655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9173e-c345-451c-b5da-50c1b27a3fea None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:31.828655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9173e-c345-451c-b5da-50c1b27a3fea None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:31.828655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9173e-c345-451c-b5da-50c1b27a3fea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:31.834754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9173e-c345-451c-b5da-50c1b27a3fea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:31.836183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9173e-c345-451c-b5da-50c1b27a3fea None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application [None req-64c9173e-c345-451c-b5da-50c1b27a3fea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 10:01:31.847796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:31.854200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:31.855342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:01:31.855342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:01:31.855342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:01:31.855342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:01:31.855342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:01:31.855342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:01:31.855342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 10:01:31.855342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: ERROR keystone.server.flask.application Aug 24 10:01:31.855342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 642/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Aug 24 10:01:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:31.855902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1610964-a1e6-44c1-8d3e-5aa5d8e0809a None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:31.909268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:31.961566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1610964-a1e6-44c1-8d3e-5aa5d8e0809a None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:31.994502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67161181-6d45-4e79-bc6d-33c327a20a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:31.995100 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161181-6d45-4e79-bc6d-33c327a20a6b None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:31.995489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161181-6d45-4e79-bc6d-33c327a20a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:31.995769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161181-6d45-4e79-bc6d-33c327a20a6b None admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:31.996439 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67161181-6d45-4e79-bc6d-33c327a20a6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:31.996774 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67161181-6d45-4e79-bc6d-33c327a20a6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:31.998865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67161181-6d45-4e79-bc6d-33c327a20a6b None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:32.012242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 644/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Aug 24 10:01:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:01:32.019154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9173e-c345-451c-b5da-50c1b27a3fea None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:32.118415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9173e-c345-451c-b5da-50c1b27a3fea None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:32.150994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38f1e4-965e-405d-bbc8-769a7611929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:32.151837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38f1e4-965e-405d-bbc8-769a7611929d None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:32.152110 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38f1e4-965e-405d-bbc8-769a7611929d None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:32.152376 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38f1e4-965e-405d-bbc8-769a7611929d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:32.153115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38f1e4-965e-405d-bbc8-769a7611929d None admin] RBAC: Adding query filter params (role.id=2a679dbfb6674ce88a681a353738cb1f, scope.project.id=80e7b34d5e4a480691bed8235d4722c8, user.id=b1aa47a883604336a8ab90d324470477) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:32.153988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38f1e4-965e-405d-bbc8-769a7611929d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:32.155394 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38f1e4-965e-405d-bbc8-769a7611929d None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:32.161020 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 643/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Aug 24 10:01:32 2022] GET /identity/v3/role_assignments?user.id=b1aa47a883604336a8ab90d324470477&scope.project.id=80e7b34d5e4a480691bed8235d4722c8&role.id=2a679dbfb6674ce88a681a353738cb1f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:01:34.103045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d712d3-2750-45ed-99fa-2a91674ea7fb None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:34.103618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d712d3-2750-45ed-99fa-2a91674ea7fb None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:34.103618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d712d3-2750-45ed-99fa-2a91674ea7fb None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:34.104960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 645/1288] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:34.110740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c17926-effb-42d0-a58c-41bab35f5de1 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:34.112959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c17926-effb-42d0-a58c-41bab35f5de1 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:34.112959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c17926-effb-42d0-a58c-41bab35f5de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:34.129853 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-22c17926-effb-42d0-a58c-41bab35f5de1 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:34.131059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-22c17926-effb-42d0-a58c-41bab35f5de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:34.162901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:34.200236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 644/1289] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:01:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:01:34.473556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d712d3-2750-45ed-99fa-2a91674ea7fb None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:34.575927 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d712d3-2750-45ed-99fa-2a91674ea7fb None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:34.586561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:34.611488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222259d-1ca4-4ba4-b2c3-77da3384a1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:34.612102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222259d-1ca4-4ba4-b2c3-77da3384a1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:34.612380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222259d-1ca4-4ba4-b2c3-77da3384a1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:34.612656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222259d-1ca4-4ba4-b2c3-77da3384a1b8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:34.613259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222259d-1ca4-4ba4-b2c3-77da3384a1b8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:34.614666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222259d-1ca4-4ba4-b2c3-77da3384a1b8 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-a222259d-1ca4-4ba4-b2c3-77da3384a1b8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:34.623480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:34.628259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:34.630242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 646/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Aug 24 10:01:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:34.633233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c17926-effb-42d0-a58c-41bab35f5de1 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:34.731942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c17926-effb-42d0-a58c-41bab35f5de1 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:34.764853 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37950ed5-0c6e-4894-a1ab-3e65356a99be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:34.765466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37950ed5-0c6e-4894-a1ab-3e65356a99be None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:34.765742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37950ed5-0c6e-4894-a1ab-3e65356a99be None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:34.766011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37950ed5-0c6e-4894-a1ab-3e65356a99be None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:34.766629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37950ed5-0c6e-4894-a1ab-3e65356a99be None admin] RBAC: Adding query filter params (name=placement) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:34.766929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37950ed5-0c6e-4894-a1ab-3e65356a99be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:34.768323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37950ed5-0c6e-4894-a1ab-3e65356a99be None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:34.773572 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 645/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Aug 24 10:01:34 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:34.780332 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222259d-1ca4-4ba4-b2c3-77da3384a1b8 None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:34.878925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222259d-1ca4-4ba4-b2c3-77da3384a1b8 None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:34.914978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe8fc7-91ee-464e-b75a-7dff61d525b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:34.915584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8fc7-91ee-464e-b75a-7dff61d525b4 None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:34.915864 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8fc7-91ee-464e-b75a-7dff61d525b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:34.916139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8fc7-91ee-464e-b75a-7dff61d525b4 None admin] PATH_INFO: `/v3/services` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:34.916762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe8fc7-91ee-464e-b75a-7dff61d525b4 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:34.917101 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe8fc7-91ee-464e-b75a-7dff61d525b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:34.918458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe8fc7-91ee-464e-b75a-7dff61d525b4 None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:34.923378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 647/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Aug 24 10:01:34 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:36.819952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1cca9-15af-47e8-94fd-b5ce37f5b33e None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:36.820543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1cca9-15af-47e8-94fd-b5ce37f5b33e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:36.820543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1cca9-15af-47e8-94fd-b5ce37f5b33e None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:36.821887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 646/1293] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:36.827849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae5a28-913a-4688-b753-432d4e9cd06f None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:36.828381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae5a28-913a-4688-b753-432d4e9cd06f None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:36.828381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae5a28-913a-4688-b753-432d4e9cd06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:36.847985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a7ae5a28-913a-4688-b753-432d4e9cd06f None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:36.849265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a7ae5a28-913a-4688-b753-432d4e9cd06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:36.880757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:36.921532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 648/1294] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:01:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:01:37.192610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1cca9-15af-47e8-94fd-b5ce37f5b33e None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:37.225081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:37.295740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1cca9-15af-47e8-94fd-b5ce37f5b33e None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:37.330621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86264a6-58b6-458a-ab22-a92f6300ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:37.331220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86264a6-58b6-458a-ab22-a92f6300ba3c None admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:37.331499 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86264a6-58b6-458a-ab22-a92f6300ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:37.331769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86264a6-58b6-458a-ab22-a92f6300ba3c None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:37.332398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86264a6-58b6-458a-ab22-a92f6300ba3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:37.333778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86264a6-58b6-458a-ab22-a92f6300ba3c None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:37.398554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 647/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Aug 24 10:01:37 2022] POST /identity/v3/services => generated 246 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:39.327132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b4c68-6562-479c-bc93-cbb1aafdee22 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:39.327650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b4c68-6562-479c-bc93-cbb1aafdee22 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:39.327756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b4c68-6562-479c-bc93-cbb1aafdee22 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:39.329101 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 649/1296] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:01:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:39.334630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae0131-9e71-4b1a-896c-5889788f4b29 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:39.335022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae0131-9e71-4b1a-896c-5889788f4b29 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:39.335185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae0131-9e71-4b1a-896c-5889788f4b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:39.353118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-bbae0131-9e71-4b1a-896c-5889788f4b29 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:39.357627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-bbae0131-9e71-4b1a-896c-5889788f4b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:39.427164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 648/1297] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:01:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:01:39.702677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b4c68-6562-479c-bc93-cbb1aafdee22 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:39.820220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b4c68-6562-479c-bc93-cbb1aafdee22 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:39.854059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b53167-7156-4db8-9158-08c5a83f882c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:39.854667 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53167-7156-4db8-9158-08c5a83f882c None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:39.854944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53167-7156-4db8-9158-08c5a83f882c None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:39.855232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53167-7156-4db8-9158-08c5a83f882c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:39.855783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b53167-7156-4db8-9158-08c5a83f882c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:39.857728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b53167-7156-4db8-9158-08c5a83f882c None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-24b53167-7156-4db8-9158-08c5a83f882c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:39.866770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:39.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:39.877742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 650/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Aug 24 10:01:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:39.878559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae0131-9e71-4b1a-896c-5889788f4b29 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:39.892692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:39.974445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae0131-9e71-4b1a-896c-5889788f4b29 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:40.008291 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26155711-e5e9-4d27-85da-4af5ce19f3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:40.008913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155711-e5e9-4d27-85da-4af5ce19f3ef None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:40.009222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155711-e5e9-4d27-85da-4af5ce19f3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:40.009491 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155711-e5e9-4d27-85da-4af5ce19f3ef None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:40.010111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26155711-e5e9-4d27-85da-4af5ce19f3ef None admin] RBAC: Adding query filter params (name=placement) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:40.010413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26155711-e5e9-4d27-85da-4af5ce19f3ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:40.011770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26155711-e5e9-4d27-85da-4af5ce19f3ef None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:40.017638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 649/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Aug 24 10:01:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:01:40.023836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b53167-7156-4db8-9158-08c5a83f882c None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:40.054703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:40.123968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b53167-7156-4db8-9158-08c5a83f882c None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:40.157567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644bc41-61a3-4871-8be2-35188bcf42ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:40.158166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644bc41-61a3-4871-8be2-35188bcf42ee None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:40.158446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644bc41-61a3-4871-8be2-35188bcf42ee None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:40.158719 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644bc41-61a3-4871-8be2-35188bcf42ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:40.159412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644bc41-61a3-4871-8be2-35188bcf42ee None admin] RBAC: Adding query filter params (interface=public, service_id=0dbaf91b070a4c4ea5b260372459f252, region_id=RegionOne) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:40.159717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644bc41-61a3-4871-8be2-35188bcf42ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:40.161103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644bc41-61a3-4871-8be2-35188bcf42ee None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:40.165926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 651/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Aug 24 10:01:40 2022] GET /identity/v3/endpoints?service_id=0dbaf91b070a4c4ea5b260372459f252&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:01:40.172449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26155711-e5e9-4d27-85da-4af5ce19f3ef None admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:40.254452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:40.271687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26155711-e5e9-4d27-85da-4af5ce19f3ef None admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:40.305731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08935ef7-93a4-4e1a-bd42-cb50263142bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:40.306332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935ef7-93a4-4e1a-bd42-cb50263142bb None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:40.306612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935ef7-93a4-4e1a-bd42-cb50263142bb None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:40.307621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935ef7-93a4-4e1a-bd42-cb50263142bb None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:40.307621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08935ef7-93a4-4e1a-bd42-cb50263142bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:40.308801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08935ef7-93a4-4e1a-bd42-cb50263142bb None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:40.316473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 650/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Aug 24 10:01:40 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:42.217029 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297e65c-30de-4281-b026-4b9a9796443c None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:42.217884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297e65c-30de-4281-b026-4b9a9796443c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:42.217884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297e65c-30de-4281-b026-4b9a9796443c None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:42.219187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 652/1302] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:42.224970 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cced6-e242-435e-82af-49a59a414d07 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:42.225415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cced6-e242-435e-82af-49a59a414d07 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:42.225511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cced6-e242-435e-82af-49a59a414d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:42.244432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-341cced6-e242-435e-82af-49a59a414d07 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:42.245641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-341cced6-e242-435e-82af-49a59a414d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:42.312350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 651/1303] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:01:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 10:01:42.581257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297e65c-30de-4281-b026-4b9a9796443c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:42.677015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:42.683711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297e65c-30de-4281-b026-4b9a9796443c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:42.717466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c62d1-4d64-42ba-8d64-8c617aec254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:42.718090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c62d1-4d64-42ba-8d64-8c617aec254f None admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:42.718371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c62d1-4d64-42ba-8d64-8c617aec254f None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:42.718646 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c62d1-4d64-42ba-8d64-8c617aec254f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:42.719221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c62d1-4d64-42ba-8d64-8c617aec254f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:42.720624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c62d1-4d64-42ba-8d64-8c617aec254f None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application [None req-1d0c62d1-4d64-42ba-8d64-8c617aec254f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:01:42.729204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:01:42.730374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application with Lock( Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: ERROR keystone.server.flask.application Aug 24 10:01:42.731790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 653/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Aug 24 10:01:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 10:01:42.736540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cced6-e242-435e-82af-49a59a414d07 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:42.812272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:42.835111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cced6-e242-435e-82af-49a59a414d07 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:42.867629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f849d3-fc36-4191-bb17-5b38432525f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:42.868228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f849d3-fc36-4191-bb17-5b38432525f0 None admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:42.868508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f849d3-fc36-4191-bb17-5b38432525f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:42.868777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f849d3-fc36-4191-bb17-5b38432525f0 None admin] PATH_INFO: `/v3/services` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:42.869427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f849d3-fc36-4191-bb17-5b38432525f0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:42.869727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f849d3-fc36-4191-bb17-5b38432525f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:42.871090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f849d3-fc36-4191-bb17-5b38432525f0 None admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:42.876573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 652/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Aug 24 10:01:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:42.882399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c62d1-4d64-42ba-8d64-8c617aec254f None admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:42.979709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c62d1-4d64-42ba-8d64-8c617aec254f None admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:43.013194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c6d37-91b9-4202-a238-9848653f5b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:43.013787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c6d37-91b9-4202-a238-9848653f5b3e None admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:43.014053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c6d37-91b9-4202-a238-9848653f5b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:43.014316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c6d37-91b9-4202-a238-9848653f5b3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:43.014971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c6d37-91b9-4202-a238-9848653f5b3e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:43.016309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c6d37-91b9-4202-a238-9848653f5b3e None admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:43.116266 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 654/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:42 2022] POST /identity/v3/endpoints => generated 331 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:52.961593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=30 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:53.111490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:42.503068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e01bd-046d-415f-9edd-c67e3f09547e None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:42.503666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e01bd-046d-415f-9edd-c67e3f09547e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:42.503666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e01bd-046d-415f-9edd-c67e3f09547e None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:42.505310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 653/1307] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:08:42.514708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233a00f-a289-493a-ad6b-9e42a2023df8 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:42.515586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233a00f-a289-493a-ad6b-9e42a2023df8 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:42.516026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233a00f-a289-493a-ad6b-9e42a2023df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:42.695603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a233a00f-a289-493a-ad6b-9e42a2023df8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:08:42.697144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a233a00f-a289-493a-ad6b-9e42a2023df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:42.954463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 655/1308] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:08:43.288777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de95a9-229e-4145-ac78-2079734bbb92 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:43.289476 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de95a9-229e-4145-ac78-2079734bbb92 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:43.289913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de95a9-229e-4145-ac78-2079734bbb92 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:43.294052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 654/1309] 10.222.0.30 () {54 vars in 965 bytes} [Wed Aug 24 10:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:08:43.298157 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb449cf-26c4-46fb-8cda-bdd91666df09 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:43.298453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb449cf-26c4-46fb-8cda-bdd91666df09 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:43.298731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb449cf-26c4-46fb-8cda-bdd91666df09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:43.395175 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-9bb449cf-26c4-46fb-8cda-bdd91666df09 None None] MFA Rules not processed for user `ae9b6525009b41288dd918437acee92a`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:08:43.396641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9bb449cf-26c4-46fb-8cda-bdd91666df09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:43.420481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:43.483828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 656/1310] 10.222.0.30 () {58 vars in 1062 bytes} [Wed Aug 24 10:08:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:08:43.492782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de95a9-229e-4145-ac78-2079734bbb92 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:43.494817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-50de95a9-229e-4145-ac78-2079734bbb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:43.614377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de95a9-229e-4145-ac78-2079734bbb92 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:43.704437 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a1ce6-1b53-4a4b-ad87-7dce2e59c001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:43.705341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a1ce6-1b53-4a4b-ad87-7dce2e59c001 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:43.705773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a1ce6-1b53-4a4b-ad87-7dce2e59c001 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:43.706220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a1ce6-1b53-4a4b-ad87-7dce2e59c001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:43.708082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b78a1ce6-1b53-4a4b-ad87-7dce2e59c001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:43.775558 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:43.838105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a1ce6-1b53-4a4b-ad87-7dce2e59c001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:43.839555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a1ce6-1b53-4a4b-ad87-7dce2e59c001 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:43.861173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 655/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:46.114249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340122f-4202-4124-ac5a-9af0dc4043b0 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:46.114807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340122f-4202-4124-ac5a-9af0dc4043b0 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:46.114807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340122f-4202-4124-ac5a-9af0dc4043b0 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:46.116976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 657/1312] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:08:46.122720 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde51d-dfe7-4e4b-bf6f-2b61aa7610a3 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:46.123218 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde51d-dfe7-4e4b-bf6f-2b61aa7610a3 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:46.123413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde51d-dfe7-4e4b-bf6f-2b61aa7610a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:46.202890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-76dde51d-dfe7-4e4b-bf6f-2b61aa7610a3 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:08:46.204110 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-76dde51d-dfe7-4e4b-bf6f-2b61aa7610a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:46.306339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 656/1313] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:08:46.606678 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420424e-9015-4e29-9797-e86625b86650 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:46.606966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420424e-9015-4e29-9797-e86625b86650 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:46.607248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420424e-9015-4e29-9797-e86625b86650 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:46.608595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 658/1314] 10.222.0.30 () {54 vars in 965 bytes} [Wed Aug 24 10:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:08:46.614670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f6f74-d3db-49c6-a6bd-21bd3b483b25 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:46.615739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f6f74-d3db-49c6-a6bd-21bd3b483b25 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:46.616248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f6f74-d3db-49c6-a6bd-21bd3b483b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:46.628391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:46.693489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-571f6f74-d3db-49c6-a6bd-21bd3b483b25 None None] MFA Rules not processed for user `ae9b6525009b41288dd918437acee92a`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:08:46.694734 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-571f6f74-d3db-49c6-a6bd-21bd3b483b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:46.769996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 657/1315] 10.222.0.30 () {58 vars in 1062 bytes} [Wed Aug 24 10:08:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:08:46.779884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420424e-9015-4e29-9797-e86625b86650 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:46.781464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9420424e-9015-4e29-9797-e86625b86650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:46.883587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420424e-9015-4e29-9797-e86625b86650 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:46.953853 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:46.956333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243c5c6-7f49-41cc-887b-d90d5e44ea5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:46.957009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c5c6-7f49-41cc-887b-d90d5e44ea5a service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:46.957327 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c5c6-7f49-41cc-887b-d90d5e44ea5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:46.957660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c5c6-7f49-41cc-887b-d90d5e44ea5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:46.960060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7243c5c6-7f49-41cc-887b-d90d5e44ea5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:47.067677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243c5c6-7f49-41cc-887b-d90d5e44ea5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:47.069707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243c5c6-7f49-41cc-887b-d90d5e44ea5a service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:47.087969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 659/1316] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:08:49.716807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfb80f-5781-4447-bd6e-8712eb958dbe None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:49.717391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfb80f-5781-4447-bd6e-8712eb958dbe None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:49.717391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfb80f-5781-4447-bd6e-8712eb958dbe None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:49.718793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 658/1317] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:08:49.725108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bd8c8-9f82-4154-ac3b-d03b33afe3b7 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:49.726044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bd8c8-9f82-4154-ac3b-d03b33afe3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:49.726331 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bd8c8-9f82-4154-ac3b-d03b33afe3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:49.791884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6e9bd8c8-9f82-4154-ac3b-d03b33afe3b7 None None] MFA Rules not processed for user `9d60ef5b46c84eaeaebaf462462afffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:08:49.793134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6e9bd8c8-9f82-4154-ac3b-d03b33afe3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:49.844602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:49.890181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 660/1318] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:08:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:08:50.175142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfb80f-5781-4447-bd6e-8712eb958dbe None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:50.176800 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ffbfb80f-5781-4447-bd6e-8712eb958dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:50.282317 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfb80f-5781-4447-bd6e-8712eb958dbe None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:50.285179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:50.357529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21873060-1bfe-4400-a4ca-1d2e7e3b18ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:50.358132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873060-1bfe-4400-a4ca-1d2e7e3b18ee service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:50.358413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873060-1bfe-4400-a4ca-1d2e7e3b18ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:50.358689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873060-1bfe-4400-a4ca-1d2e7e3b18ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:50.360298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-21873060-1bfe-4400-a4ca-1d2e7e3b18ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:50.471809 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873060-1bfe-4400-a4ca-1d2e7e3b18ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:50.473190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873060-1bfe-4400-a4ca-1d2e7e3b18ee service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:50.491955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 659/1319] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:53.281268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a176b5-ed99-45e0-9d73-53be0270f191 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:53.281809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a176b5-ed99-45e0-9d73-53be0270f191 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:53.281809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a176b5-ed99-45e0-9d73-53be0270f191 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:53.283176 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 661/1320] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:08:53.289270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d47b0-f145-44c1-994c-9412f57bc33a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:53.289747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d47b0-f145-44c1-994c-9412f57bc33a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:53.289932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d47b0-f145-44c1-994c-9412f57bc33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:53.339555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:53.365597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-f65d47b0-f145-44c1-994c-9412f57bc33a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:08:53.367509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f65d47b0-f145-44c1-994c-9412f57bc33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:53.469174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 660/1321] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:08:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:08:53.756213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a176b5-ed99-45e0-9d73-53be0270f191 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:53.758236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-82a176b5-ed99-45e0-9d73-53be0270f191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:53.860055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a176b5-ed99-45e0-9d73-53be0270f191 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:53.931827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b1fce-4d60-4106-9fed-7ed578ceac50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:53.932430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b1fce-4d60-4106-9fed-7ed578ceac50 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:53.932712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b1fce-4d60-4106-9fed-7ed578ceac50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:53.933019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b1fce-4d60-4106-9fed-7ed578ceac50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:53.934628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1e6b1fce-4d60-4106-9fed-7ed578ceac50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:53.941992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:54.044741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b1fce-4d60-4106-9fed-7ed578ceac50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:54.046165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b1fce-4d60-4106-9fed-7ed578ceac50 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:54.064784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 662/1322] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:57.093639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176418e-dd8c-4af3-9587-44aca95b3714 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:57.094202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176418e-dd8c-4af3-9587-44aca95b3714 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:57.094202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176418e-dd8c-4af3-9587-44aca95b3714 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:57.095546 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 661/1323] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:08:57.101361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15293847-a63d-457b-b9ad-5a6875942516 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:57.101770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15293847-a63d-457b-b9ad-5a6875942516 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:57.101921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15293847-a63d-457b-b9ad-5a6875942516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:57.170170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-15293847-a63d-457b-b9ad-5a6875942516 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:08:57.171400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-15293847-a63d-457b-b9ad-5a6875942516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:57.230350 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:57.269829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 663/1324] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:08:57.564020 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176418e-dd8c-4af3-9587-44aca95b3714 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:57.565718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f176418e-dd8c-4af3-9587-44aca95b3714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:57.673250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176418e-dd8c-4af3-9587-44aca95b3714 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:57.685614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:57.748989 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6d86e-d55e-4ae6-9d6e-813e5825aebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:57.749592 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6d86e-d55e-4ae6-9d6e-813e5825aebf service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:57.749866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6d86e-d55e-4ae6-9d6e-813e5825aebf service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:57.750135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6d86e-d55e-4ae6-9d6e-813e5825aebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:57.751731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c1f6d86e-d55e-4ae6-9d6e-813e5825aebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:57.869452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6d86e-d55e-4ae6-9d6e-813e5825aebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:57.870909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6d86e-d55e-4ae6-9d6e-813e5825aebf service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:57.888593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 662/1325] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:59.885704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d94ace-f33d-4f08-9539-9130510c8aaa None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:59.886217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d94ace-f33d-4f08-9539-9130510c8aaa None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:59.886217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d94ace-f33d-4f08-9539-9130510c8aaa None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:59.887573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 664/1326] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:08:59.893271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141122a-64c2-471d-a177-189f987a386b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:59.893710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141122a-64c2-471d-a177-189f987a386b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:59.893883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141122a-64c2-471d-a177-189f987a386b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:59.943517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:59.962129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-4141122a-64c2-471d-a177-189f987a386b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:08:59.963347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4141122a-64c2-471d-a177-189f987a386b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:00.063166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 663/1327] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:00.345520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d94ace-f33d-4f08-9539-9130510c8aaa None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:00.347112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-59d94ace-f33d-4f08-9539-9130510c8aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:00.452217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d94ace-f33d-4f08-9539-9130510c8aaa None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:00.539293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ba7c9-a13d-4e0f-be98-cdfe8ed0afb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:00.544779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ba7c9-a13d-4e0f-be98-cdfe8ed0afb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:00.544779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ba7c9-a13d-4e0f-be98-cdfe8ed0afb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:00.544779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ba7c9-a13d-4e0f-be98-cdfe8ed0afb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:00.544779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c53ba7c9-a13d-4e0f-be98-cdfe8ed0afb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:00.560630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:00.656247 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ba7c9-a13d-4e0f-be98-cdfe8ed0afb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:00.657842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ba7c9-a13d-4e0f-be98-cdfe8ed0afb3 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:00.676340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 665/1328] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:02.638615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935792b-8c1f-4d69-b38c-e0238d71a00b None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:02.639188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935792b-8c1f-4d69-b38c-e0238d71a00b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:02.639188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935792b-8c1f-4d69-b38c-e0238d71a00b None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:02.640747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 664/1329] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:02.647091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac276a97-fda9-482c-b923-46372b11129a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:02.647525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac276a97-fda9-482c-b923-46372b11129a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:02.647668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac276a97-fda9-482c-b923-46372b11129a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:02.717601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-ac276a97-fda9-482c-b923-46372b11129a None None] MFA Rules not processed for user `9d60ef5b46c84eaeaebaf462462afffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:02.718810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ac276a97-fda9-482c-b923-46372b11129a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:02.789523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:02.821014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 666/1330] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:03.106983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935792b-8c1f-4d69-b38c-e0238d71a00b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:03.108577 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4935792b-8c1f-4d69-b38c-e0238d71a00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:03.240880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935792b-8c1f-4d69-b38c-e0238d71a00b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:03.270131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:03.319444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3307d01-eb4c-49bb-888e-9b3f628b5d44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:03.320047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3307d01-eb4c-49bb-888e-9b3f628b5d44 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:03.320322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3307d01-eb4c-49bb-888e-9b3f628b5d44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:03.320591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3307d01-eb4c-49bb-888e-9b3f628b5d44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:03.322229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b3307d01-eb4c-49bb-888e-9b3f628b5d44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:03.433051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3307d01-eb4c-49bb-888e-9b3f628b5d44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:03.434421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3307d01-eb4c-49bb-888e-9b3f628b5d44 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:03.452253 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 665/1331] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:03 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:05.968366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf1b0d-37c3-446d-87ba-bee82243fbdf None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:05.969069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf1b0d-37c3-446d-87ba-bee82243fbdf None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:05.969069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf1b0d-37c3-446d-87ba-bee82243fbdf None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:05.970275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 667/1332] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:09:05.976395 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b18b19-cc7f-41d4-898a-a94614f4e9e6 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:05.976837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b18b19-cc7f-41d4-898a-a94614f4e9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:05.980086 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b18b19-cc7f-41d4-898a-a94614f4e9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:06.043395 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:06.046108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e1b18b19-cc7f-41d4-898a-a94614f4e9e6 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:06.047338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e1b18b19-cc7f-41d4-898a-a94614f4e9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:06.143852 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 666/1333] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:06.431117 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf1b0d-37c3-446d-87ba-bee82243fbdf None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:06.432696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5faf1b0d-37c3-446d-87ba-bee82243fbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:06.540921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf1b0d-37c3-446d-87ba-bee82243fbdf None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:06.617581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c17793-63bc-4804-ae45-be562bf48bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:06.618189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17793-63bc-4804-ae45-be562bf48bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:06.618468 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17793-63bc-4804-ae45-be562bf48bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:06.618743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17793-63bc-4804-ae45-be562bf48bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:06.620372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-55c17793-63bc-4804-ae45-be562bf48bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:06.650872 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:06.734598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c17793-63bc-4804-ae45-be562bf48bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:06.736026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c17793-63bc-4804-ae45-be562bf48bf8 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:06.754070 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 668/1334] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:08.760392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2153b-264f-4870-8432-6a82f7a181a0 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:08.761012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2153b-264f-4870-8432-6a82f7a181a0 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:08.761012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2153b-264f-4870-8432-6a82f7a181a0 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:08.762322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 667/1335] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:08.767944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64107ceb-a29a-4b64-afb4-10e7035bf995 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:08.768365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64107ceb-a29a-4b64-afb4-10e7035bf995 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:08.768507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64107ceb-a29a-4b64-afb4-10e7035bf995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:08.834838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-64107ceb-a29a-4b64-afb4-10e7035bf995 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:08.836058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-64107ceb-a29a-4b64-afb4-10e7035bf995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:08.914695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:08.936278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 669/1336] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:09.220689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2153b-264f-4870-8432-6a82f7a181a0 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:09.225560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5db2153b-264f-4870-8432-6a82f7a181a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:09.340432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2153b-264f-4870-8432-6a82f7a181a0 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:09.372547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:09.420972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9eba8a-6741-4c21-8710-d2790906a8ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:09.421584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9eba8a-6741-4c21-8710-d2790906a8ac service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:09.421863 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9eba8a-6741-4c21-8710-d2790906a8ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:09.422133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9eba8a-6741-4c21-8710-d2790906a8ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:09.424565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ef9eba8a-6741-4c21-8710-d2790906a8ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:09.544933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9eba8a-6741-4c21-8710-d2790906a8ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:09.546299 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9eba8a-6741-4c21-8710-d2790906a8ac service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:09.572881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 668/1337] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:12.259618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bdc2c-7817-4263-9838-29c463f65ef6 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:12.260164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bdc2c-7817-4263-9838-29c463f65ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:12.260164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bdc2c-7817-4263-9838-29c463f65ef6 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:12.261560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 670/1338] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:12.267156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f84fff-8d25-45ed-9bf0-e10a4293ad30 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:12.267629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f84fff-8d25-45ed-9bf0-e10a4293ad30 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:12.267735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f84fff-8d25-45ed-9bf0-e10a4293ad30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:12.337452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b1f84fff-8d25-45ed-9bf0-e10a4293ad30 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:12.338697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b1f84fff-8d25-45ed-9bf0-e10a4293ad30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:12.346905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:12.436085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 669/1339] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:12.721033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bdc2c-7817-4263-9838-29c463f65ef6 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:12.722577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8c2bdc2c-7817-4263-9838-29c463f65ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:12.824736 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bdc2c-7817-4263-9838-29c463f65ef6 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:12.898224 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfde9c-0a7d-493e-8db4-819bc03b700f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:12.898822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfde9c-0a7d-493e-8db4-819bc03b700f service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:12.899109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfde9c-0a7d-493e-8db4-819bc03b700f service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:12.899382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfde9c-0a7d-493e-8db4-819bc03b700f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:12.900997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d4dfde9c-0a7d-493e-8db4-819bc03b700f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:12.936305 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:13.010729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfde9c-0a7d-493e-8db4-819bc03b700f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:13.012076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfde9c-0a7d-493e-8db4-819bc03b700f service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:13.030775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 671/1340] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:17.284398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb3b5f-7ed3-476a-a393-191a5c6a7ff6 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:17.285017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb3b5f-7ed3-476a-a393-191a5c6a7ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:17.285202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb3b5f-7ed3-476a-a393-191a5c6a7ff6 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:17.286544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 670/1341] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:17.292050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff987961-c6c5-42e2-bb6b-181b2fb33d0c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:17.292507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff987961-c6c5-42e2-bb6b-181b2fb33d0c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:17.292608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff987961-c6c5-42e2-bb6b-181b2fb33d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:17.362614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-ff987961-c6c5-42e2-bb6b-181b2fb33d0c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:17.363902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ff987961-c6c5-42e2-bb6b-181b2fb33d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:17.450229 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:17.462526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 672/1342] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:09:17.762986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb3b5f-7ed3-476a-a393-191a5c6a7ff6 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:17.764529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-45eb3b5f-7ed3-476a-a393-191a5c6a7ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:17.875320 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb3b5f-7ed3-476a-a393-191a5c6a7ff6 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:17.914288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:17.950258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e7178-3ab6-4822-aaa0-49031c3ba87b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:17.950865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e7178-3ab6-4822-aaa0-49031c3ba87b service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:17.951144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e7178-3ab6-4822-aaa0-49031c3ba87b service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:17.951417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e7178-3ab6-4822-aaa0-49031c3ba87b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:17.953062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5f1e7178-3ab6-4822-aaa0-49031c3ba87b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:18.069100 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e7178-3ab6-4822-aaa0-49031c3ba87b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:18.070487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e7178-3ab6-4822-aaa0-49031c3ba87b service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:18.091380 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 671/1343] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:20.993919 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ea8e1-852d-4e2f-b14b-405dfe823491 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:20.994478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ea8e1-852d-4e2f-b14b-405dfe823491 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:20.994478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ea8e1-852d-4e2f-b14b-405dfe823491 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:20.995709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 673/1344] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:21.000897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f62a3-39ce-4ab4-9dc4-be8af3f3d00a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:21.001332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f62a3-39ce-4ab4-9dc4-be8af3f3d00a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:21.001440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f62a3-39ce-4ab4-9dc4-be8af3f3d00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:21.069108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-301f62a3-39ce-4ab4-9dc4-be8af3f3d00a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:21.070309 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-301f62a3-39ce-4ab4-9dc4-be8af3f3d00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:21.088974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:21.169357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 672/1345] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:21.459752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ea8e1-852d-4e2f-b14b-405dfe823491 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:21.461317 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b27ea8e1-852d-4e2f-b14b-405dfe823491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:21.565364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ea8e1-852d-4e2f-b14b-405dfe823491 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:21.645098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97f73a-6f06-4234-a219-70f7b74405cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:21.645357 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97f73a-6f06-4234-a219-70f7b74405cb service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:21.645633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97f73a-6f06-4234-a219-70f7b74405cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:21.645909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97f73a-6f06-4234-a219-70f7b74405cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:21.647532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-dc97f73a-6f06-4234-a219-70f7b74405cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:21.694107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:21.769280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97f73a-6f06-4234-a219-70f7b74405cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:21.771129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97f73a-6f06-4234-a219-70f7b74405cb service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:21.790701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 674/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:26.028957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9b18-1263-4642-9b38-aa264ac5170b None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:26.029484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9b18-1263-4642-9b38-aa264ac5170b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:26.029597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9b18-1263-4642-9b38-aa264ac5170b None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:26.030864 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 673/1347] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:26.037195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931e862-1b3f-4d05-9c71-2961a60d50ed None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:26.037695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931e862-1b3f-4d05-9c71-2961a60d50ed None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:26.037773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931e862-1b3f-4d05-9c71-2961a60d50ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:26.109733 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-3931e862-1b3f-4d05-9c71-2961a60d50ed None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:26.111008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3931e862-1b3f-4d05-9c71-2961a60d50ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:26.215264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:26.218624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 675/1348] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:09:26.506065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a9b18-1263-4642-9b38-aa264ac5170b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:26.507461 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-109a9b18-1263-4642-9b38-aa264ac5170b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:26.626621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a9b18-1263-4642-9b38-aa264ac5170b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:26.675134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:26.701971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3124e9-1301-4472-bc9d-e165f98364e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:26.702575 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3124e9-1301-4472-bc9d-e165f98364e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:26.702852 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3124e9-1301-4472-bc9d-e165f98364e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:26.703122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3124e9-1301-4472-bc9d-e165f98364e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:26.704723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4b3124e9-1301-4472-bc9d-e165f98364e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:26.821780 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3124e9-1301-4472-bc9d-e165f98364e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:26.823131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3124e9-1301-4472-bc9d-e165f98364e7 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:26.841866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 674/1349] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:31.903124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bfb37-c345-4b98-8777-2e159e5de9f6 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:31.903611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bfb37-c345-4b98-8777-2e159e5de9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:31.903611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bfb37-c345-4b98-8777-2e159e5de9f6 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:31.905326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 676/1350] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:31.911271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6120f6d-c6db-4563-9378-abf269aee47a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:31.911742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6120f6d-c6db-4563-9378-abf269aee47a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:31.911824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6120f6d-c6db-4563-9378-abf269aee47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:32.016423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-d6120f6d-c6db-4563-9378-abf269aee47a None None] MFA Rules not processed for user `b1aa47a883604336a8ab90d324470477`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:32.017896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d6120f6d-c6db-4563-9378-abf269aee47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:32.040029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:32.191947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 675/1351] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:09:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:36.341332 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ad52-a501-47f1-a246-950e769b1f0d None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:36.342230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ad52-a501-47f1-a246-950e769b1f0d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:36.342534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ad52-a501-47f1-a246-950e769b1f0d None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:36.344141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 677/1352] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:09:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:36.349931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db19e41-7927-415f-9b54-3be336ed1339 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:36.350878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db19e41-7927-415f-9b54-3be336ed1339 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:36.351471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db19e41-7927-415f-9b54-3be336ed1339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:36.450234 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-4db19e41-7927-415f-9b54-3be336ed1339 None None] MFA Rules not processed for user `b1aa47a883604336a8ab90d324470477`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:36.451587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4db19e41-7927-415f-9b54-3be336ed1339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:36.530924 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:36.567089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 676/1353] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:09:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:38.056555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599c30-9b95-448a-85a2-a13091582fec None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:38.057130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599c30-9b95-448a-85a2-a13091582fec None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:38.057130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599c30-9b95-448a-85a2-a13091582fec None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:38.058467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 678/1354] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:38.063809 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75673e8-a821-4aac-9114-d9ca85878f74 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:38.064235 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75673e8-a821-4aac-9114-d9ca85878f74 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:38.064371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75673e8-a821-4aac-9114-d9ca85878f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:38.156965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e75673e8-a821-4aac-9114-d9ca85878f74 None None] MFA Rules not processed for user `b1aa47a883604336a8ab90d324470477`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:38.158560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e75673e8-a821-4aac-9114-d9ca85878f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:38.275693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 677/1355] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:09:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:09:41.590609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637c22f-74d4-48fc-95de-efd9f119c4e7 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:41.590609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637c22f-74d4-48fc-95de-efd9f119c4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:41.590609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637c22f-74d4-48fc-95de-efd9f119c4e7 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:41.590609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 679/1356] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:41.593726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20823449-37e8-432d-a0f3-1a21cf50cf35 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:41.594124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20823449-37e8-432d-a0f3-1a21cf50cf35 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:41.594278 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20823449-37e8-432d-a0f3-1a21cf50cf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:41.635796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:41.686462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-20823449-37e8-432d-a0f3-1a21cf50cf35 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:41.688020 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-20823449-37e8-432d-a0f3-1a21cf50cf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:41.856923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 678/1357] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:42.162029 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c6e0a-3707-453e-8def-4751404288be None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:42.162309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c6e0a-3707-453e-8def-4751404288be None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:42.162751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c6e0a-3707-453e-8def-4751404288be None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:42.164084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 680/1358] 10.222.0.30 () {54 vars in 962 bytes} [Wed Aug 24 10:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:42.172295 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46f333-671b-4d28-a90e-542f9fe47ebb None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:42.173030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46f333-671b-4d28-a90e-542f9fe47ebb None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:42.173400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46f333-671b-4d28-a90e-542f9fe47ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:42.264661 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-8c46f333-671b-4d28-a90e-542f9fe47ebb None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:42.265925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8c46f333-671b-4d28-a90e-542f9fe47ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:42.324594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:42.396370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 679/1359] 10.222.0.30 () {58 vars in 1059 bytes} [Wed Aug 24 10:09:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:42.405158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c6e0a-3707-453e-8def-4751404288be None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:42.407207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a95c6e0a-3707-453e-8def-4751404288be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:42.570101 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c6e0a-3707-453e-8def-4751404288be None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:42.650455 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:42.691279 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027627d7-c60c-475f-a8b6-59d7eae5c65d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:42.691981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027627d7-c60c-475f-a8b6-59d7eae5c65d service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:42.692498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027627d7-c60c-475f-a8b6-59d7eae5c65d service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:42.692852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027627d7-c60c-475f-a8b6-59d7eae5c65d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:42.695155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-027627d7-c60c-475f-a8b6-59d7eae5c65d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:42.789510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027627d7-c60c-475f-a8b6-59d7eae5c65d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:42.791271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027627d7-c60c-475f-a8b6-59d7eae5c65d service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:42.810571 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 681/1360] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:44.979325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0aa08-a6cb-4fba-83cd-4a5835627d14 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:44.980192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0aa08-a6cb-4fba-83cd-4a5835627d14 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:44.980621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0aa08-a6cb-4fba-83cd-4a5835627d14 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:44.982183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 680/1361] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:44.988174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86891dfa-62b4-4c69-ac71-a17752fff51d None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:44.988727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86891dfa-62b4-4c69-ac71-a17752fff51d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:44.988727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86891dfa-62b4-4c69-ac71-a17752fff51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:45.060977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-86891dfa-62b4-4c69-ac71-a17752fff51d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:45.062579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-86891dfa-62b4-4c69-ac71-a17752fff51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:45.103601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:45.173219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 682/1362] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:45.470037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0aa08-a6cb-4fba-83cd-4a5835627d14 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:45.471605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e3c0aa08-a6cb-4fba-83cd-4a5835627d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:45.636636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0aa08-a6cb-4fba-83cd-4a5835627d14 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:45.648781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:45.765043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615f10c-3ed6-4f37-92ef-753656687522 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:45.765561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f10c-3ed6-4f37-92ef-753656687522 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:45.765838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f10c-3ed6-4f37-92ef-753656687522 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:45.766109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f10c-3ed6-4f37-92ef-753656687522 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:45.767727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2615f10c-3ed6-4f37-92ef-753656687522 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:45.873889 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615f10c-3ed6-4f37-92ef-753656687522 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:45.875662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615f10c-3ed6-4f37-92ef-753656687522 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:45.898784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 681/1363] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:48.348666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023ef9c-ea0a-46e9-8a5e-c79343099cdc None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:48.349218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023ef9c-ea0a-46e9-8a5e-c79343099cdc None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:48.349300 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023ef9c-ea0a-46e9-8a5e-c79343099cdc None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:48.350826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 683/1364] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:48.358002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f7422-4732-4637-9049-415f144c6d0f None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:48.358475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f7422-4732-4637-9049-415f144c6d0f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:48.358582 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f7422-4732-4637-9049-415f144c6d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:48.377094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:48.431610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-888f7422-4732-4637-9049-415f144c6d0f None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:48.432845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-888f7422-4732-4637-9049-415f144c6d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:48.544447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 682/1365] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:48.850708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295c846-2902-41e4-bf83-ba004ffbb3cb None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:48.851029 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295c846-2902-41e4-bf83-ba004ffbb3cb None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:48.851275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295c846-2902-41e4-bf83-ba004ffbb3cb None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:48.852757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 684/1366] 10.222.0.30 () {54 vars in 962 bytes} [Wed Aug 24 10:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:48.858275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2b9a9-6143-4350-872a-c4cae0c6b9bb None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:48.858560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2b9a9-6143-4350-872a-c4cae0c6b9bb None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:48.858840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2b9a9-6143-4350-872a-c4cae0c6b9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:48.948506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-f3e2b9a9-6143-4350-872a-c4cae0c6b9bb None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:48.949777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f3e2b9a9-6143-4350-872a-c4cae0c6b9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:48.991634 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:49.086293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 683/1367] 10.222.0.30 () {58 vars in 1059 bytes} [Wed Aug 24 10:09:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:49.094623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295c846-2902-41e4-bf83-ba004ffbb3cb None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:49.097585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c295c846-2902-41e4-bf83-ba004ffbb3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:49.258561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295c846-2902-41e4-bf83-ba004ffbb3cb None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:49.281123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:49.385768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad814e24-4e7e-4581-8533-f2b6bda44436 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:49.386433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad814e24-4e7e-4581-8533-f2b6bda44436 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:49.386663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad814e24-4e7e-4581-8533-f2b6bda44436 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:49.386898 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad814e24-4e7e-4581-8533-f2b6bda44436 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:49.388478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ad814e24-4e7e-4581-8533-f2b6bda44436 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:49.481881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad814e24-4e7e-4581-8533-f2b6bda44436 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:49.483483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad814e24-4e7e-4581-8533-f2b6bda44436 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:49.502199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 685/1368] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:49.935577 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b7fcf-5341-47a5-a849-c605c3254321 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:49.936137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b7fcf-5341-47a5-a849-c605c3254321 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:49.936684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b7fcf-5341-47a5-a849-c605c3254321 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:49.944904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62a07f-b05a-46ea-b4e6-1f7f0dd2ade2 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:49.945209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62a07f-b05a-46ea-b4e6-1f7f0dd2ade2 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:49.945588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62a07f-b05a-46ea-b4e6-1f7f0dd2ade2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:49.950066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 684/1369] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:49.981557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:50.023203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-4b62a07f-b05a-46ea-b4e6-1f7f0dd2ade2 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:50.024420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-4b62a07f-b05a-46ea-b4e6-1f7f0dd2ade2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:50.133201 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 686/1370] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:09:50.728122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d294bfa-1090-4c6c-95f7-c9751cfb2c44 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:50.728532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d294bfa-1090-4c6c-95f7-c9751cfb2c44 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:50.728957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d294bfa-1090-4c6c-95f7-c9751cfb2c44 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:50.730551 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 685/1371] 10.222.0.30 () {54 vars in 964 bytes} [Wed Aug 24 10:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:50.735621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f6cc8-7c6d-427f-8113-187e9dd33d28 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:50.736069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f6cc8-7c6d-427f-8113-187e9dd33d28 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:50.736149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f6cc8-7c6d-427f-8113-187e9dd33d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:50.821062 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a22f6cc8-7c6d-427f-8113-187e9dd33d28 None None] MFA Rules not processed for user `c71783d1327a47149a74414218dd7219`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:50.822266 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a22f6cc8-7c6d-427f-8113-187e9dd33d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:50.872356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:50.907645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 687/1372] 10.222.0.30 () {58 vars in 1061 bytes} [Wed Aug 24 10:09:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:50.917098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d294bfa-1090-4c6c-95f7-c9751cfb2c44 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:50.918786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6d294bfa-1090-4c6c-95f7-c9751cfb2c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:51.038681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d294bfa-1090-4c6c-95f7-c9751cfb2c44 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:51.080441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:51.122657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c0093-fc7d-47e2-94e1-cdfd6e7f9d4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:51.123505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c0093-fc7d-47e2-94e1-cdfd6e7f9d4d service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:51.124264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c0093-fc7d-47e2-94e1-cdfd6e7f9d4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:51.124978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c0093-fc7d-47e2-94e1-cdfd6e7f9d4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:51.128033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7c0c0093-fc7d-47e2-94e1-cdfd6e7f9d4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:51.242895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c0093-fc7d-47e2-94e1-cdfd6e7f9d4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:51.244327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c0093-fc7d-47e2-94e1-cdfd6e7f9d4d service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:51.263874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 686/1373] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:09:51.774074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6688efe-6108-4d60-b1df-a8407a247016 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:51.774647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6688efe-6108-4d60-b1df-a8407a247016 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:51.774647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6688efe-6108-4d60-b1df-a8407a247016 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:51.776040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 688/1374] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:51.781506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ddd70-ef75-4a6d-b970-6c297ddd5c26 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:51.781797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ddd70-ef75-4a6d-b970-6c297ddd5c26 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:51.782073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ddd70-ef75-4a6d-b970-6c297ddd5c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:51.852493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-401ddd70-ef75-4a6d-b970-6c297ddd5c26 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:51.853744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-401ddd70-ef75-4a6d-b970-6c297ddd5c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:51.875439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:51.960406 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 687/1375] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:52.261934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6688efe-6108-4d60-b1df-a8407a247016 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:52.263728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c6688efe-6108-4d60-b1df-a8407a247016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:52.419177 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6688efe-6108-4d60-b1df-a8407a247016 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:52.471755 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:52.535980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8850d-64fb-4a22-a786-967847bf50c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:52.536592 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8850d-64fb-4a22-a786-967847bf50c4 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:52.536894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8850d-64fb-4a22-a786-967847bf50c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:52.537292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8850d-64fb-4a22-a786-967847bf50c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:52.538911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-27b8850d-64fb-4a22-a786-967847bf50c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:52.629071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8850d-64fb-4a22-a786-967847bf50c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:52.630686 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8850d-64fb-4a22-a786-967847bf50c4 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:52.651173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 689/1376] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:54.770144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368e37a-20cf-492d-89dc-cc1e34cfc562 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:54.770686 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368e37a-20cf-492d-89dc-cc1e34cfc562 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:54.770686 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368e37a-20cf-492d-89dc-cc1e34cfc562 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:54.772040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 688/1377] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:54.777271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63496c24-b80c-451f-814a-6e35f507f19b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:54.777682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63496c24-b80c-451f-814a-6e35f507f19b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:54.777829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63496c24-b80c-451f-814a-6e35f507f19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:54.848438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-63496c24-b80c-451f-814a-6e35f507f19b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:54.849679 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-63496c24-b80c-451f-814a-6e35f507f19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:54.856309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:54.960166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 690/1378] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:55.277070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368e37a-20cf-492d-89dc-cc1e34cfc562 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:55.278586 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3368e37a-20cf-492d-89dc-cc1e34cfc562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:55.441814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368e37a-20cf-492d-89dc-cc1e34cfc562 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:55.446318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:55.568692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8dbe2a-2d73-4314-ae18-a17b9a5b46ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:55.569609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8dbe2a-2d73-4314-ae18-a17b9a5b46ce service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:55.570165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8dbe2a-2d73-4314-ae18-a17b9a5b46ce service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:55.570642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8dbe2a-2d73-4314-ae18-a17b9a5b46ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:55.572411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-bf8dbe2a-2d73-4314-ae18-a17b9a5b46ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:55.673114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8dbe2a-2d73-4314-ae18-a17b9a5b46ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:55.675737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8dbe2a-2d73-4314-ae18-a17b9a5b46ce service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:55.697180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 689/1379] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:57.802432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febc1a1-445e-47ff-9752-20d20a3a15a2 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:57.803000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febc1a1-445e-47ff-9752-20d20a3a15a2 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:57.803000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febc1a1-445e-47ff-9752-20d20a3a15a2 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:57.804526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 691/1380] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:09:57.809878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768cabf-9969-42af-b47c-dd1d30606ce4 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:57.810302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768cabf-9969-42af-b47c-dd1d30606ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:57.810528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768cabf-9969-42af-b47c-dd1d30606ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:57.822743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:57.885086 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-a768cabf-9969-42af-b47c-dd1d30606ce4 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:57.886312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a768cabf-9969-42af-b47c-dd1d30606ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:58.020952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 690/1381] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:58.313527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febc1a1-445e-47ff-9752-20d20a3a15a2 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:58.315337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6febc1a1-445e-47ff-9752-20d20a3a15a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:58.461953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:58.475182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febc1a1-445e-47ff-9752-20d20a3a15a2 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:58.602761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30397c61-b0fd-433c-a50b-9962f30b428c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:58.603400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30397c61-b0fd-433c-a50b-9962f30b428c service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:58.603685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30397c61-b0fd-433c-a50b-9962f30b428c service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:58.603960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30397c61-b0fd-433c-a50b-9962f30b428c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:58.605611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-30397c61-b0fd-433c-a50b-9962f30b428c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:58.708724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30397c61-b0fd-433c-a50b-9962f30b428c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:58.710797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30397c61-b0fd-433c-a50b-9962f30b428c service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:58.724227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:58.733220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 692/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:00.948235 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740adc8-34cf-4c08-96f7-8c56e51766d5 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:00.948925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740adc8-34cf-4c08-96f7-8c56e51766d5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:00.948925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740adc8-34cf-4c08-96f7-8c56e51766d5 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:00.950141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 691/1383] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:00.955559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f829bb-3c34-45a8-a2b6-f45cfd2969ca None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:00.956101 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f829bb-3c34-45a8-a2b6-f45cfd2969ca None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:00.956200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f829bb-3c34-45a8-a2b6-f45cfd2969ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:01.036134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c1f829bb-3c34-45a8-a2b6-f45cfd2969ca None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:01.037447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c1f829bb-3c34-45a8-a2b6-f45cfd2969ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:01.148931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 693/1384] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:01.437148 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740adc8-34cf-4c08-96f7-8c56e51766d5 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:01.438714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e740adc8-34cf-4c08-96f7-8c56e51766d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:01.541176 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:01.603547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740adc8-34cf-4c08-96f7-8c56e51766d5 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:01.732732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f3277-7aa6-4d6b-a6f7-c5bb08e2ab22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:01.733370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f3277-7aa6-4d6b-a6f7-c5bb08e2ab22 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:01.733658 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f3277-7aa6-4d6b-a6f7-c5bb08e2ab22 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:01.733935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f3277-7aa6-4d6b-a6f7-c5bb08e2ab22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:01.735670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cd5f3277-7aa6-4d6b-a6f7-c5bb08e2ab22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:01.794157 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:01.835635 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f3277-7aa6-4d6b-a6f7-c5bb08e2ab22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:01.837281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f3277-7aa6-4d6b-a6f7-c5bb08e2ab22 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:01.856198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 692/1385] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:04.164662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b01cc-5511-4bf8-a693-71c014f9b006 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:04.165611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b01cc-5511-4bf8-a693-71c014f9b006 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:04.165611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b01cc-5511-4bf8-a693-71c014f9b006 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:04.166724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 694/1386] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:04.172708 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba26ec-f434-47bd-9c0d-c1f449633d6e None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:04.173327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba26ec-f434-47bd-9c0d-c1f449633d6e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:04.173327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba26ec-f434-47bd-9c0d-c1f449633d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:04.244841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0aba26ec-f434-47bd-9c0d-c1f449633d6e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:04.246080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0aba26ec-f434-47bd-9c0d-c1f449633d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:04.373737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 693/1387] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:10:04.687399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b01cc-5511-4bf8-a693-71c014f9b006 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:04.689435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a12b01cc-5511-4bf8-a693-71c014f9b006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:04.785603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:04.864059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b01cc-5511-4bf8-a693-71c014f9b006 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:04.988626 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c0cf1-f165-47bb-bcf6-98db5b6a1e22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:04.989242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c0cf1-f165-47bb-bcf6-98db5b6a1e22 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:04.989531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c0cf1-f165-47bb-bcf6-98db5b6a1e22 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:04.989792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c0cf1-f165-47bb-bcf6-98db5b6a1e22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:04.991845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-177c0cf1-f165-47bb-bcf6-98db5b6a1e22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:05.031822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:05.106740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c0cf1-f165-47bb-bcf6-98db5b6a1e22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:05.110507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c0cf1-f165-47bb-bcf6-98db5b6a1e22 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:05.147743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 695/1388] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:07.275014 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e51a-1219-4af2-8b02-eaac0d20207a None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:07.275842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e51a-1219-4af2-8b02-eaac0d20207a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:07.275842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e51a-1219-4af2-8b02-eaac0d20207a None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:07.277896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 694/1389] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:10:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:07.286624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c628fa9-58d4-4ca8-b4cb-e9bdfacf7339 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:07.287183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c628fa9-58d4-4ca8-b4cb-e9bdfacf7339 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:07.287452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c628fa9-58d4-4ca8-b4cb-e9bdfacf7339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:07.350616 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3f27-19be-412b-a6d5-efe312064144 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:07.350950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3f27-19be-412b-a6d5-efe312064144 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:07.352000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3f27-19be-412b-a6d5-efe312064144 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:07.352553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 695/1390] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:10:07.358765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a2b65-bf16-4f03-8bb3-481e73ea087c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:07.359303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a2b65-bf16-4f03-8bb3-481e73ea087c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:07.359652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a2b65-bf16-4f03-8bb3-481e73ea087c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:07.365666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-8c628fa9-58d4-4ca8-b4cb-e9bdfacf7339 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:07.366947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8c628fa9-58d4-4ca8-b4cb-e9bdfacf7339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:07.407619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:07.442680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e34a2b65-bf16-4f03-8bb3-481e73ea087c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:07.443681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e34a2b65-bf16-4f03-8bb3-481e73ea087c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:07.481034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 696/1391] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:07.556587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 696/1392] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:10:07.785799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c628fa9-58d4-4ca8-b4cb-e9bdfacf7339 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:07.786992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8c628fa9-58d4-4ca8-b4cb-e9bdfacf7339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:07.806927 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:07.867667 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe45fc-2289-496a-97a9-946a6f6473a7 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:07.867952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe45fc-2289-496a-97a9-946a6f6473a7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:07.868221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe45fc-2289-496a-97a9-946a6f6473a7 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:07.869891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 697/1393] 10.222.0.30 () {54 vars in 967 bytes} [Wed Aug 24 10:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:10:07.874909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e41581-9e8f-412e-8e88-08a45bb71b66 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:07.875198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e41581-9e8f-412e-8e88-08a45bb71b66 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:07.875478 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e41581-9e8f-412e-8e88-08a45bb71b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:07.933254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c628fa9-58d4-4ca8-b4cb-e9bdfacf7339 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:07.964876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e5e41581-9e8f-412e-8e88-08a45bb71b66 None None] MFA Rules not processed for user `d0b124ca267b4a4391557ad1396524cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:07.966169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e5e41581-9e8f-412e-8e88-08a45bb71b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:08.023991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:08.060123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 698/1394] 10.222.0.30 () {58 vars in 1064 bytes} [Wed Aug 24 10:10:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:10:08.069172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e41581-9e8f-412e-8e88-08a45bb71b66 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:08.076934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e41581-9e8f-412e-8e88-08a45bb71b66 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:08.078435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a5156-9106-4d41-a7a0-c508dfc19f4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:08.078435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a5156-9106-4d41-a7a0-c508dfc19f4b service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:08.078435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a5156-9106-4d41-a7a0-c508dfc19f4b service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:08.078435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a5156-9106-4d41-a7a0-c508dfc19f4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:08.079889 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-194a5156-9106-4d41-a7a0-c508dfc19f4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:08.088929 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:08.162334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd805ac-b825-4da7-862c-94b94fe3ce00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0b124ca267b4a4391557ad1396524cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:08.163230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd805ac-b825-4da7-862c-94b94fe3ce00 service glance] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:08.163544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd805ac-b825-4da7-862c-94b94fe3ce00 service glance] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:08.163855 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd805ac-b825-4da7-862c-94b94fe3ce00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:08.165680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-dbd805ac-b825-4da7-862c-94b94fe3ce00 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:08.183363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a5156-9106-4d41-a7a0-c508dfc19f4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:08.184932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a5156-9106-4d41-a7a0-c508dfc19f4b service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:08.205435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 697/1395] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:08.291832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd805ac-b825-4da7-862c-94b94fe3ce00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:08.293326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd805ac-b825-4da7-862c-94b94fe3ce00 service glance] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:08.309044 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:08.315075 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 699/1396] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:10:09.346611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc860e-343e-4bc7-bcf9-a06203873313 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:09.351684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc860e-343e-4bc7-bcf9-a06203873313 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:09.351684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc860e-343e-4bc7-bcf9-a06203873313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:09.427697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-88dc860e-343e-4bc7-bcf9-a06203873313 None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:09.429067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-88dc860e-343e-4bc7-bcf9-a06203873313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:09.524590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:09.528227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 698/1397] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:10:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:09.576997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920af0-c439-45a5-bd5a-547be356247c None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:09.577569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920af0-c439-45a5-bd5a-547be356247c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:09.577569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920af0-c439-45a5-bd5a-547be356247c None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:09.579061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 700/1398] 10.222.0.30 () {54 vars in 943 bytes} [Wed Aug 24 10:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:09.587820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d45f1-7f63-47bd-872f-68503c654be5 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:09.588337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d45f1-7f63-47bd-872f-68503c654be5 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:09.588821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d45f1-7f63-47bd-872f-68503c654be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:09.684311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-540d45f1-7f63-47bd-872f-68503c654be5 None None] MFA Rules not processed for user `88fe81f7b4c04e9f9a26b430b39ccd36`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:09.685737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-540d45f1-7f63-47bd-872f-68503c654be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:09.820727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 699/1399] 10.222.0.30 () {58 vars in 1040 bytes} [Wed Aug 24 10:10:09 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:09.831304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4920af0-c439-45a5-bd5a-547be356247c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:09.833008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a4920af0-c439-45a5-bd5a-547be356247c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:09.994287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4920af0-c439-45a5-bd5a-547be356247c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:10.085837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:10.109292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03390fd-612f-4bd4-a9eb-20b08b104723 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:10.118412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03390fd-612f-4bd4-a9eb-20b08b104723 service swift] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:10.118412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03390fd-612f-4bd4-a9eb-20b08b104723 service swift] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:10.118412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03390fd-612f-4bd4-a9eb-20b08b104723 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:10.118412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d03390fd-612f-4bd4-a9eb-20b08b104723 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:10.202587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03390fd-612f-4bd4-a9eb-20b08b104723 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:10.204265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03390fd-612f-4bd4-a9eb-20b08b104723 service swift] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:10.213387 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 701/1400] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:10:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:10:10.427255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390315c9-934d-4e90-9a4a-f88222a0c9ee None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:10.427815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390315c9-934d-4e90-9a4a-f88222a0c9ee None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:10.427909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390315c9-934d-4e90-9a4a-f88222a0c9ee None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:10.429330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 700/1401] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:10.434661 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d823a0-d3c7-4fda-b4c9-605e32db5987 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:10.439109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d823a0-d3c7-4fda-b4c9-605e32db5987 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:10.439109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d823a0-d3c7-4fda-b4c9-605e32db5987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:10.505198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-d0d823a0-d3c7-4fda-b4c9-605e32db5987 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:10.506442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d0d823a0-d3c7-4fda-b4c9-605e32db5987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:10.579522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:10.611413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 702/1402] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:10.905361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390315c9-934d-4e90-9a4a-f88222a0c9ee None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:10.906922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-390315c9-934d-4e90-9a4a-f88222a0c9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:10.973799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:11.091509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390315c9-934d-4e90-9a4a-f88222a0c9ee None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:11.234188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a725ce-bef4-473b-99d0-18dd4458dab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:11.234188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a725ce-bef4-473b-99d0-18dd4458dab5 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:11.234486 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a725ce-bef4-473b-99d0-18dd4458dab5 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:11.234571 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a725ce-bef4-473b-99d0-18dd4458dab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:11.236162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e9a725ce-bef4-473b-99d0-18dd4458dab5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:11.257664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:11.343110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a725ce-bef4-473b-99d0-18dd4458dab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:11.345784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a725ce-bef4-473b-99d0-18dd4458dab5 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:11.371014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 701/1403] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:13.561729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987f6cb-60d6-4607-9454-3f61b4c40e83 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:13.562247 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987f6cb-60d6-4607-9454-3f61b4c40e83 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:13.562247 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987f6cb-60d6-4607-9454-3f61b4c40e83 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:13.563647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 703/1404] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:13.570050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24b92b-faaa-487e-9771-cf2062479764 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:13.570917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24b92b-faaa-487e-9771-cf2062479764 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:13.571429 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24b92b-faaa-487e-9771-cf2062479764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:13.649951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-de24b92b-faaa-487e-9771-cf2062479764 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:13.652888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-de24b92b-faaa-487e-9771-cf2062479764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:13.760826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:13.768134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 702/1405] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:14.064308 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987f6cb-60d6-4607-9454-3f61b4c40e83 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:14.065872 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f987f6cb-60d6-4607-9454-3f61b4c40e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:14.235473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987f6cb-60d6-4607-9454-3f61b4c40e83 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:14.314375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:14.388110 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7447f9-6dae-4abd-af83-cfd9c1353da3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:14.388720 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7447f9-6dae-4abd-af83-cfd9c1353da3 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:14.389055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7447f9-6dae-4abd-af83-cfd9c1353da3 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:14.389325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7447f9-6dae-4abd-af83-cfd9c1353da3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:14.391251 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4c7447f9-6dae-4abd-af83-cfd9c1353da3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:14.522164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7447f9-6dae-4abd-af83-cfd9c1353da3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:14.523797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7447f9-6dae-4abd-af83-cfd9c1353da3 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:14.544358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 704/1406] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:17.564959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d57b5-8109-4b47-b0c6-c8595eee9245 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:17.565515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d57b5-8109-4b47-b0c6-c8595eee9245 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:17.565515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d57b5-8109-4b47-b0c6-c8595eee9245 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:17.567253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 703/1407] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:17.572119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c8416-7e25-4866-9712-0a7690e426bc None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:17.572563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c8416-7e25-4866-9712-0a7690e426bc None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:17.572661 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c8416-7e25-4866-9712-0a7690e426bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:17.650757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-c28c8416-7e25-4866-9712-0a7690e426bc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:17.652115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c28c8416-7e25-4866-9712-0a7690e426bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:17.667294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:17.768916 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 705/1408] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:10:18.074079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d57b5-8109-4b47-b0c6-c8595eee9245 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:18.075655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-655d57b5-8109-4b47-b0c6-c8595eee9245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:18.252136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d57b5-8109-4b47-b0c6-c8595eee9245 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:18.337490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:18.396798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900361a-2d36-4c41-afba-1481263ac507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:18.397433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900361a-2d36-4c41-afba-1481263ac507 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:18.397776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900361a-2d36-4c41-afba-1481263ac507 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:18.398097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900361a-2d36-4c41-afba-1481263ac507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:18.400059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1900361a-2d36-4c41-afba-1481263ac507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:18.513783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900361a-2d36-4c41-afba-1481263ac507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:18.515381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900361a-2d36-4c41-afba-1481263ac507 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:18.536902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 704/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:27.766493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=38 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:28.534518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=64 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:34.055510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b7e00-b5a9-460c-9727-348c2442083f None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:34.056041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b7e00-b5a9-460c-9727-348c2442083f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:34.056041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b7e00-b5a9-460c-9727-348c2442083f None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:34.057366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 706/1410] 10.222.0.30 () {54 vars in 918 bytes} [Wed Aug 24 10:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:34.063674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f22081-6164-4e4f-ab65-639fe081283e None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:34.064192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f22081-6164-4e4f-ab65-639fe081283e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:34.064299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f22081-6164-4e4f-ab65-639fe081283e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:34.142278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-10f22081-6164-4e4f-ab65-639fe081283e None None] MFA Rules not processed for user `b1aa47a883604336a8ab90d324470477`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:34.143488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-10f22081-6164-4e4f-ab65-639fe081283e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:34.257621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 705/1411] 10.222.0.30 () {58 vars in 1015 bytes} [Wed Aug 24 10:10:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:34.273790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73c291-c2fc-4d02-979d-4ef70a4b5cce None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:34.274078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73c291-c2fc-4d02-979d-4ef70a4b5cce None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:34.274434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73c291-c2fc-4d02-979d-4ef70a4b5cce None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:34.275777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 707/1412] 10.222.0.30 () {54 vars in 918 bytes} [Wed Aug 24 10:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:34.280613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b54ff-b9b8-4d01-a77b-ba83929f4687 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:34.280967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b54ff-b9b8-4d01-a77b-ba83929f4687 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:34.281361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b54ff-b9b8-4d01-a77b-ba83929f4687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:34.356560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:34.360208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-318b54ff-b9b8-4d01-a77b-ba83929f4687 None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:34.361464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-318b54ff-b9b8-4d01-a77b-ba83929f4687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:34.480479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 706/1413] 10.222.0.30 () {58 vars in 1015 bytes} [Wed Aug 24 10:10:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:36.922920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78439482-5707-4404-bb3a-7352fa1b1e95 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:36.923514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78439482-5707-4404-bb3a-7352fa1b1e95 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:36.923514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78439482-5707-4404-bb3a-7352fa1b1e95 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:36.924818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:10:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:10:36.930441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ed716-5c1f-4336-aac4-abd7aa2413ed None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:36.930927 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ed716-5c1f-4336-aac4-abd7aa2413ed None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:36.931037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ed716-5c1f-4336-aac4-abd7aa2413ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:36.997471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c45ed716-5c1f-4336-aac4-abd7aa2413ed None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:36.998691 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c45ed716-5c1f-4336-aac4-abd7aa2413ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:37.054014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:37.095850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:37.392790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78439482-5707-4404-bb3a-7352fa1b1e95 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:37.394374 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-78439482-5707-4404-bb3a-7352fa1b1e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:37.546483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78439482-5707-4404-bb3a-7352fa1b1e95 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:37.632037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:37.674312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f95bb8-b98c-4a07-97b8-6a519a02dcc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:37.674922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f95bb8-b98c-4a07-97b8-6a519a02dcc3 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:37.675202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f95bb8-b98c-4a07-97b8-6a519a02dcc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:37.675474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f95bb8-b98c-4a07-97b8-6a519a02dcc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:37.677131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-18f95bb8-b98c-4a07-97b8-6a519a02dcc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:37.768539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f95bb8-b98c-4a07-97b8-6a519a02dcc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:37.770171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f95bb8-b98c-4a07-97b8-6a519a02dcc3 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:37.788927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 709/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:38.699724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfa2b0-85ca-426f-a1d1-2a5581c04f16 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:38.700271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfa2b0-85ca-426f-a1d1-2a5581c04f16 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:38.700271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfa2b0-85ca-426f-a1d1-2a5581c04f16 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:38.701664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:38.707491 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da75dd7-0f88-47c7-9a8d-c4e52e44d151 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:38.707913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da75dd7-0f88-47c7-9a8d-c4e52e44d151 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:38.708029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da75dd7-0f88-47c7-9a8d-c4e52e44d151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:38.785846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-4da75dd7-0f88-47c7-9a8d-c4e52e44d151 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:38.787062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4da75dd7-0f88-47c7-9a8d-c4e52e44d151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:38.815279 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:38.886371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:39.035353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfa2b0-85ca-426f-a1d1-2a5581c04f16 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:39.036897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-84dfa2b0-85ca-426f-a1d1-2a5581c04f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:39.187039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfa2b0-85ca-426f-a1d1-2a5581c04f16 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:39.228709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:39.306246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc9bc2-3a61-4e33-92c0-2163482abc77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:39.306857 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc9bc2-3a61-4e33-92c0-2163482abc77 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:39.307145 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc9bc2-3a61-4e33-92c0-2163482abc77 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:39.307444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc9bc2-3a61-4e33-92c0-2163482abc77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:39.309100 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6ddc9bc2-3a61-4e33-92c0-2163482abc77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:39.403257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc9bc2-3a61-4e33-92c0-2163482abc77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:39.404884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc9bc2-3a61-4e33-92c0-2163482abc77 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:39.425100 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 709/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:40.326535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400cb07-5530-47d9-8cab-8d39a8c6e3ec None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:40.327032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400cb07-5530-47d9-8cab-8d39a8c6e3ec None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:40.327143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400cb07-5530-47d9-8cab-8d39a8c6e3ec None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:40.328637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:10:40.333621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb80e7-ca96-4808-a6b8-b45505cfa10b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:40.334042 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb80e7-ca96-4808-a6b8-b45505cfa10b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:40.334147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb80e7-ca96-4808-a6b8-b45505cfa10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:40.390988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:40.412169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-31eb80e7-ca96-4808-a6b8-b45505cfa10b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:40.413399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-31eb80e7-ca96-4808-a6b8-b45505cfa10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:40.510763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:40.667694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400cb07-5530-47d9-8cab-8d39a8c6e3ec None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:40.669451 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-1400cb07-5530-47d9-8cab-8d39a8c6e3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:40.825198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400cb07-5530-47d9-8cab-8d39a8c6e3ec None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:40.839424 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:40.944658 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9aab1-28d4-4181-9606-3030c3d0dd5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:40.945343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9aab1-28d4-4181-9606-3030c3d0dd5a service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:40.945642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9aab1-28d4-4181-9606-3030c3d0dd5a service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:40.945914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9aab1-28d4-4181-9606-3030c3d0dd5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:40.947696 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0cc9aab1-28d4-4181-9606-3030c3d0dd5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:41.040030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9aab1-28d4-4181-9606-3030c3d0dd5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:41.041670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9aab1-28d4-4181-9606-3030c3d0dd5a service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:41.059562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 712/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:41.941207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce0e1e-31a4-4932-8b06-eef75621af9a None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:41.941742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce0e1e-31a4-4932-8b06-eef75621af9a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:41.941742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce0e1e-31a4-4932-8b06-eef75621af9a None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:41.943051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:41.948328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966fbe0-b7ab-46d5-9b71-e3ca37429eb5 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:41.948744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966fbe0-b7ab-46d5-9b71-e3ca37429eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:41.948920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966fbe0-b7ab-46d5-9b71-e3ca37429eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:41.970070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:42.030204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-7966fbe0-b7ab-46d5-9b71-e3ca37429eb5 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:42.032049 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7966fbe0-b7ab-46d5-9b71-e3ca37429eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:42.130599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:42.284598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce0e1e-31a4-4932-8b06-eef75621af9a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:42.286221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-64ce0e1e-31a4-4932-8b06-eef75621af9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:42.402076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:42.438839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce0e1e-31a4-4932-8b06-eef75621af9a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:42.554448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f14925-905b-4d85-9578-ec721e0be0fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:42.554940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f14925-905b-4d85-9578-ec721e0be0fd service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:42.555217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f14925-905b-4d85-9578-ec721e0be0fd service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:42.555486 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f14925-905b-4d85-9578-ec721e0be0fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:42.557113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-91f14925-905b-4d85-9578-ec721e0be0fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:42.635039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:42.649000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f14925-905b-4d85-9578-ec721e0be0fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:42.650601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f14925-905b-4d85-9578-ec721e0be0fd service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:42.669331 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 712/1425] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:43.558194 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e2c3-1498-4ed2-bd14-2bf0e0390bfd None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:43.558733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e2c3-1498-4ed2-bd14-2bf0e0390bfd None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:43.558733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e2c3-1498-4ed2-bd14-2bf0e0390bfd None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:43.560152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:43.566017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098407b2-e307-4241-88a6-aa083c3fe8ac None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:43.566466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098407b2-e307-4241-88a6-aa083c3fe8ac None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:43.566625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098407b2-e307-4241-88a6-aa083c3fe8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:43.645995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-098407b2-e307-4241-88a6-aa083c3fe8ac None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:43.647197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-098407b2-e307-4241-88a6-aa083c3fe8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:43.747161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:43.899648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245e2c3-1498-4ed2-bd14-2bf0e0390bfd None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:43.901306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d245e2c3-1498-4ed2-bd14-2bf0e0390bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:43.989931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:44.053412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245e2c3-1498-4ed2-bd14-2bf0e0390bfd None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:44.169656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253804fc-3dbe-4d53-bd5c-1095c2ea77df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:44.170177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253804fc-3dbe-4d53-bd5c-1095c2ea77df service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:44.170454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253804fc-3dbe-4d53-bd5c-1095c2ea77df service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:44.170725 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253804fc-3dbe-4d53-bd5c-1095c2ea77df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:44.172790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-253804fc-3dbe-4d53-bd5c-1095c2ea77df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:44.222977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:44.274234 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253804fc-3dbe-4d53-bd5c-1095c2ea77df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:44.275880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253804fc-3dbe-4d53-bd5c-1095c2ea77df service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:44.295295 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 715/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:45.263809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd0ede-8ad6-444d-ac93-8be073e500e5 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:45.263809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd0ede-8ad6-444d-ac93-8be073e500e5 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:45.264415 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd0ede-8ad6-444d-ac93-8be073e500e5 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:45.265465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:45.270739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0385da-07a0-469b-b818-40016b053d07 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:45.271184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0385da-07a0-469b-b818-40016b053d07 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:45.271293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0385da-07a0-469b-b818-40016b053d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:45.348842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-af0385da-07a0-469b-b818-40016b053d07 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:45.350078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-af0385da-07a0-469b-b818-40016b053d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:45.453411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:45.605677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd0ede-8ad6-444d-ac93-8be073e500e5 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:45.607225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-51dd0ede-8ad6-444d-ac93-8be073e500e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:45.657187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:45.755084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd0ede-8ad6-444d-ac93-8be073e500e5 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:45.867772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e7734-2f16-4268-8dd9-e1b92983e497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:45.868292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e7734-2f16-4268-8dd9-e1b92983e497 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:45.868567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e7734-2f16-4268-8dd9-e1b92983e497 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:45.868833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e7734-2f16-4268-8dd9-e1b92983e497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:45.870431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-673e7734-2f16-4268-8dd9-e1b92983e497 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:45.878586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:45.963315 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e7734-2f16-4268-8dd9-e1b92983e497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:45.964940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e7734-2f16-4268-8dd9-e1b92983e497 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:45.983381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 715/1431] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:46.881654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4bdc2-f610-48cd-a507-4640bf23b666 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:46.882263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4bdc2-f610-48cd-a507-4640bf23b666 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:46.882263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4bdc2-f610-48cd-a507-4640bf23b666 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:46.883557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:46.888881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7584a94-94eb-4872-b8ea-d4b2db5e2ac1 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:46.889299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7584a94-94eb-4872-b8ea-d4b2db5e2ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:46.889467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7584a94-94eb-4872-b8ea-d4b2db5e2ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:46.966769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-d7584a94-94eb-4872-b8ea-d4b2db5e2ac1 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:46.968014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d7584a94-94eb-4872-b8ea-d4b2db5e2ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:47.036636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:47.066671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:47.221403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4bdc2-f610-48cd-a507-4640bf23b666 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:47.222835 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-bca4bdc2-f610-48cd-a507-4640bf23b666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:47.240442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:47.373051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4bdc2-f610-48cd-a507-4640bf23b666 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:47.462983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:47.488904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530652a5-a5f2-472f-9f04-f4d736a9561c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:47.489427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530652a5-a5f2-472f-9f04-f4d736a9561c service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:47.489710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530652a5-a5f2-472f-9f04-f4d736a9561c service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:47.489981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530652a5-a5f2-472f-9f04-f4d736a9561c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:47.491586 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-530652a5-a5f2-472f-9f04-f4d736a9561c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:47.587795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530652a5-a5f2-472f-9f04-f4d736a9561c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:47.589420 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530652a5-a5f2-472f-9f04-f4d736a9561c service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:47.607361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 718/1434] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:48.616043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db829b57-7101-4452-adae-10b05d81089c None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:48.616598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db829b57-7101-4452-adae-10b05d81089c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:48.616598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db829b57-7101-4452-adae-10b05d81089c None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:48.617975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:48.623085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095e808-2162-4a90-8276-da07a07b742d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:48.623506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095e808-2162-4a90-8276-da07a07b742d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:48.623644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095e808-2162-4a90-8276-da07a07b742d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:48.702128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-7095e808-2162-4a90-8276-da07a07b742d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:48.703962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7095e808-2162-4a90-8276-da07a07b742d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:48.747053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:48.804389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:48.958509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db829b57-7101-4452-adae-10b05d81089c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:48.960061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-db829b57-7101-4452-adae-10b05d81089c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:49.107579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db829b57-7101-4452-adae-10b05d81089c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:49.161739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:49.220655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dec7a-228c-4827-b024-58c8c45bc2ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:49.221304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dec7a-228c-4827-b024-58c8c45bc2ef service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:49.221574 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dec7a-228c-4827-b024-58c8c45bc2ef service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:49.221848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dec7a-228c-4827-b024-58c8c45bc2ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:49.223477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8f8dec7a-228c-4827-b024-58c8c45bc2ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:49.316368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8dec7a-228c-4827-b024-58c8c45bc2ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:49.318218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8dec7a-228c-4827-b024-58c8c45bc2ef service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:49.336399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 718/1437] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:50.245381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228b0c9-9184-49d7-99e3-8b8bf4225e03 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:50.245955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228b0c9-9184-49d7-99e3-8b8bf4225e03 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:50.245955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228b0c9-9184-49d7-99e3-8b8bf4225e03 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:50.247325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:50.252648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d72b4-5e6b-4784-a4fa-2f2625828b15 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:50.253104 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d72b4-5e6b-4784-a4fa-2f2625828b15 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:50.253230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d72b4-5e6b-4784-a4fa-2f2625828b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:50.330472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-404d72b4-5e6b-4784-a4fa-2f2625828b15 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:50.331684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-404d72b4-5e6b-4784-a4fa-2f2625828b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:50.333685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:50.427950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:50.581021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228b0c9-9184-49d7-99e3-8b8bf4225e03 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:50.582537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b228b0c9-9184-49d7-99e3-8b8bf4225e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:50.728122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228b0c9-9184-49d7-99e3-8b8bf4225e03 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:50.756004 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:50.844785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015bc23-a0ae-4b19-a6bc-97c0a273d4bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:50.845333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015bc23-a0ae-4b19-a6bc-97c0a273d4bc service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:50.845611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015bc23-a0ae-4b19-a6bc-97c0a273d4bc service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:50.845883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015bc23-a0ae-4b19-a6bc-97c0a273d4bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:50.847498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b015bc23-a0ae-4b19-a6bc-97c0a273d4bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:50.938358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015bc23-a0ae-4b19-a6bc-97c0a273d4bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:50.939984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015bc23-a0ae-4b19-a6bc-97c0a273d4bc service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:50.958513 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 721/1440] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:52.792402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c211b-f6cd-48a9-84bb-4af45ebb0f85 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:52.792977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c211b-f6cd-48a9-84bb-4af45ebb0f85 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:52.793099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c211b-f6cd-48a9-84bb-4af45ebb0f85 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:52.794597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:52.800932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007bfb5-80be-4064-89ce-8cc87ca12c00 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:52.801485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007bfb5-80be-4064-89ce-8cc87ca12c00 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:52.801485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007bfb5-80be-4064-89ce-8cc87ca12c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:52.843428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:52.869573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-1007bfb5-80be-4064-89ce-8cc87ca12c00 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:52.870815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-1007bfb5-80be-4064-89ce-8cc87ca12c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:52.971342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:53.262315 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c211b-f6cd-48a9-84bb-4af45ebb0f85 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:53.263847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a69c211b-f6cd-48a9-84bb-4af45ebb0f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:53.369266 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c211b-f6cd-48a9-84bb-4af45ebb0f85 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:53.403256 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:53.447199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664ab8a-9119-426d-8568-52c1c22d4f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:53.447811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664ab8a-9119-426d-8568-52c1c22d4f2a service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:53.448091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664ab8a-9119-426d-8568-52c1c22d4f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:53.448365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664ab8a-9119-426d-8568-52c1c22d4f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:53.450035 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5664ab8a-9119-426d-8568-52c1c22d4f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:53.562013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664ab8a-9119-426d-8568-52c1c22d4f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:53.563420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664ab8a-9119-426d-8568-52c1c22d4f2a service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:53.584360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 721/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:55.703660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215919be-588f-4fc0-9b6f-56c60fab3f84 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:55.704256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215919be-588f-4fc0-9b6f-56c60fab3f84 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:55.704256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215919be-588f-4fc0-9b6f-56c60fab3f84 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:55.705687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:10:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:55.711182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66110768-de5c-488f-a1bf-8c4cdedc5d29 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:55.716920 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66110768-de5c-488f-a1bf-8c4cdedc5d29 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:55.716920 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66110768-de5c-488f-a1bf-8c4cdedc5d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:55.783357 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-66110768-de5c-488f-a1bf-8c4cdedc5d29 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:55.784586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-66110768-de5c-488f-a1bf-8c4cdedc5d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:55.797523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:55.882514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:56.176646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215919be-588f-4fc0-9b6f-56c60fab3f84 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:56.178212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-215919be-588f-4fc0-9b6f-56c60fab3f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:56.286675 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215919be-588f-4fc0-9b6f-56c60fab3f84 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:56.301133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:56.364301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bc233-d073-42cc-b400-6faee063710b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:56.364944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bc233-d073-42cc-b400-6faee063710b service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:56.365225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bc233-d073-42cc-b400-6faee063710b service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:56.365496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bc233-d073-42cc-b400-6faee063710b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:56.367105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c12bc233-d073-42cc-b400-6faee063710b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:56.478263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bc233-d073-42cc-b400-6faee063710b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:56.479627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bc233-d073-42cc-b400-6faee063710b service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:56.498131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 724/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:58.835893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bbb37-3104-4ba1-9171-2ac18c27eb12 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:58.836416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bbb37-3104-4ba1-9171-2ac18c27eb12 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:58.836416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bbb37-3104-4ba1-9171-2ac18c27eb12 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:58.837745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:10:58.843096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1e029-327e-4645-8520-acea70ad9884 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:58.843516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1e029-327e-4645-8520-acea70ad9884 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:58.843631 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1e029-327e-4645-8520-acea70ad9884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:58.894042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:58.911737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-10a1e029-327e-4645-8520-acea70ad9884 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:58.912981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-10a1e029-327e-4645-8520-acea70ad9884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:59.012085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:59.300913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bbb37-3104-4ba1-9171-2ac18c27eb12 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:59.302449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-799bbb37-3104-4ba1-9171-2ac18c27eb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:59.408540 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bbb37-3104-4ba1-9171-2ac18c27eb12 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:59.448903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:59.482932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573addf-1e22-49da-b4e7-8fcc94ed4c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:59.483554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573addf-1e22-49da-b4e7-8fcc94ed4c78 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:59.483830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573addf-1e22-49da-b4e7-8fcc94ed4c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:59.484097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573addf-1e22-49da-b4e7-8fcc94ed4c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:59.485711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e573addf-1e22-49da-b4e7-8fcc94ed4c78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:59.598083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573addf-1e22-49da-b4e7-8fcc94ed4c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:59.599502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573addf-1e22-49da-b4e7-8fcc94ed4c78 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:59.618371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 724/1449] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:01.761334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a8c8-9b84-4f53-83bb-fa49ee52ffed None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:01.761915 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a8c8-9b84-4f53-83bb-fa49ee52ffed None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:01.761915 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a8c8-9b84-4f53-83bb-fa49ee52ffed None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:01.763247 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:01.769107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af19e04-c4e9-4d51-b574-c0d97d3f191b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:01.769595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af19e04-c4e9-4d51-b574-c0d97d3f191b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:01.769710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af19e04-c4e9-4d51-b574-c0d97d3f191b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:01.841085 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6af19e04-c4e9-4d51-b574-c0d97d3f191b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:01.842863 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6af19e04-c4e9-4d51-b574-c0d97d3f191b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:01.864319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:01.942341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:02.239319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53a8c8-9b84-4f53-83bb-fa49ee52ffed None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:02.241055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-fe53a8c8-9b84-4f53-83bb-fa49ee52ffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:02.349896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53a8c8-9b84-4f53-83bb-fa49ee52ffed None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:02.369748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:02.425572 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfc2d3-ee12-44a1-af76-c223c42740d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:02.426174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfc2d3-ee12-44a1-af76-c223c42740d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:02.426452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfc2d3-ee12-44a1-af76-c223c42740d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:02.426722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfc2d3-ee12-44a1-af76-c223c42740d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:02.428396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-31bfc2d3-ee12-44a1-af76-c223c42740d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:02.538729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfc2d3-ee12-44a1-af76-c223c42740d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:02.540080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfc2d3-ee12-44a1-af76-c223c42740d9 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:02.557933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 727/1452] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:04.572342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a2ef7-3dc2-46a4-b97f-851c5ac9109f None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:04.572823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a2ef7-3dc2-46a4-b97f-851c5ac9109f None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:04.572953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a2ef7-3dc2-46a4-b97f-851c5ac9109f None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:04.574275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:04.580153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e618-2172-43fa-a192-6b30bbae3624 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:04.580612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e618-2172-43fa-a192-6b30bbae3624 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:04.580722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e618-2172-43fa-a192-6b30bbae3624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:04.645706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:04.648362 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-cd21e618-2172-43fa-a192-6b30bbae3624 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:04.649613 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cd21e618-2172-43fa-a192-6b30bbae3624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:04.746963 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:05.040702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a2ef7-3dc2-46a4-b97f-851c5ac9109f None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:05.042275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b61a2ef7-3dc2-46a4-b97f-851c5ac9109f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:05.146725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a2ef7-3dc2-46a4-b97f-851c5ac9109f None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:05.195476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:05.221257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94afd0d-26ca-4d0e-8874-c06a747fded5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:05.221863 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94afd0d-26ca-4d0e-8874-c06a747fded5 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:05.222144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94afd0d-26ca-4d0e-8874-c06a747fded5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:05.222417 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94afd0d-26ca-4d0e-8874-c06a747fded5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:05.224010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b94afd0d-26ca-4d0e-8874-c06a747fded5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:05.334395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94afd0d-26ca-4d0e-8874-c06a747fded5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:05.335837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94afd0d-26ca-4d0e-8874-c06a747fded5 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:05.354935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 727/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:07.362427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1966ad5-7302-4c99-8452-351c94de92d8 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:07.363072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1966ad5-7302-4c99-8452-351c94de92d8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:07.363072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1966ad5-7302-4c99-8452-351c94de92d8 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:07.364314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:07.369785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396bddb-45f3-4aa5-ab15-267bdbafc34d None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:07.370255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396bddb-45f3-4aa5-ab15-267bdbafc34d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:07.370371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396bddb-45f3-4aa5-ab15-267bdbafc34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:07.438362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-3396bddb-45f3-4aa5-ab15-267bdbafc34d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:07.439561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3396bddb-45f3-4aa5-ab15-267bdbafc34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:07.468830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:07.536915 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:07.830833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1966ad5-7302-4c99-8452-351c94de92d8 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:07.832388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f1966ad5-7302-4c99-8452-351c94de92d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:07.937159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1966ad5-7302-4c99-8452-351c94de92d8 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:07.964926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:08.012585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674e088-11fa-4731-a3f4-4f647b23d542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:08.013244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674e088-11fa-4731-a3f4-4f647b23d542 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:08.013517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674e088-11fa-4731-a3f4-4f647b23d542 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:08.014308 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674e088-11fa-4731-a3f4-4f647b23d542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:08.015917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f674e088-11fa-4731-a3f4-4f647b23d542 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:08.125264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674e088-11fa-4731-a3f4-4f647b23d542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:08.126702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674e088-11fa-4731-a3f4-4f647b23d542 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:08.145263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 730/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:10.213614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f60b14-7d8e-4f3b-b996-bcf27126c20e None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:10.214188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f60b14-7d8e-4f3b-b996-bcf27126c20e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:10.214188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f60b14-7d8e-4f3b-b996-bcf27126c20e None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:10.215516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:11:10.221030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce4045-9beb-4d95-bd1c-ac454363fe5b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:10.221475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce4045-9beb-4d95-bd1c-ac454363fe5b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:10.221598 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce4045-9beb-4d95-bd1c-ac454363fe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:10.289795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ffce4045-9beb-4d95-bd1c-ac454363fe5b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:10.291022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ffce4045-9beb-4d95-bd1c-ac454363fe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:10.301877 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:10.393412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:10.682982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f60b14-7d8e-4f3b-b996-bcf27126c20e None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:10.684649 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f9f60b14-7d8e-4f3b-b996-bcf27126c20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:10.788189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f60b14-7d8e-4f3b-b996-bcf27126c20e None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:10.844278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:10.863302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf53a6-894c-457a-98b4-ba884afda9b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:10.863911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf53a6-894c-457a-98b4-ba884afda9b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:10.864193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf53a6-894c-457a-98b4-ba884afda9b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:10.864538 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf53a6-894c-457a-98b4-ba884afda9b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:10.866115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-09bf53a6-894c-457a-98b4-ba884afda9b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:10.976025 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf53a6-894c-457a-98b4-ba884afda9b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:10.977521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf53a6-894c-457a-98b4-ba884afda9b4 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:10.997045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 730/1461] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:12.994443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67861af3-2c42-424f-8069-657008d2cbb8 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:12.994984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67861af3-2c42-424f-8069-657008d2cbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:12.995085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67861af3-2c42-424f-8069-657008d2cbb8 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:12.996386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:13.001905 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a455b8-5e9b-41c3-a775-152b2399f4f3 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:13.002350 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a455b8-5e9b-41c3-a775-152b2399f4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:13.002449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a455b8-5e9b-41c3-a775-152b2399f4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:13.069642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-b0a455b8-5e9b-41c3-a775-152b2399f4f3 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:13.070863 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b0a455b8-5e9b-41c3-a775-152b2399f4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:13.111191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:13.291062 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:13.581364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67861af3-2c42-424f-8069-657008d2cbb8 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:13.582956 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-67861af3-2c42-424f-8069-657008d2cbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:13.688627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67861af3-2c42-424f-8069-657008d2cbb8 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:13.725750 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:13.763803 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38220f-0178-401d-b63e-264a58a3e9f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:13.764416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38220f-0178-401d-b63e-264a58a3e9f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:13.764698 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38220f-0178-401d-b63e-264a58a3e9f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:13.765024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38220f-0178-401d-b63e-264a58a3e9f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:13.766660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ea38220f-0178-401d-b63e-264a58a3e9f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:13.879993 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38220f-0178-401d-b63e-264a58a3e9f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:13.881470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38220f-0178-401d-b63e-264a58a3e9f2 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:13.904677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 733/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:15.846437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11485a51-541a-48d2-9f38-fb725bd38510 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:15.847009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11485a51-541a-48d2-9f38-fb725bd38510 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:15.847009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11485a51-541a-48d2-9f38-fb725bd38510 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:15.848379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:15.853696 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a0455-f3ef-4ef9-92c0-5bdca635ce60 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:15.854104 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a0455-f3ef-4ef9-92c0-5bdca635ce60 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:15.854250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a0455-f3ef-4ef9-92c0-5bdca635ce60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:15.923059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-9b5a0455-f3ef-4ef9-92c0-5bdca635ce60 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:15.924286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9b5a0455-f3ef-4ef9-92c0-5bdca635ce60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:15.942975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:16.023226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:16.313284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11485a51-541a-48d2-9f38-fb725bd38510 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:16.314805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-11485a51-541a-48d2-9f38-fb725bd38510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:16.417886 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11485a51-541a-48d2-9f38-fb725bd38510 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:16.485108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:16.492006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3fee6-1679-4bee-8e99-3b7761dc7141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:16.492615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3fee6-1679-4bee-8e99-3b7761dc7141 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:16.492921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3fee6-1679-4bee-8e99-3b7761dc7141 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:16.493201 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3fee6-1679-4bee-8e99-3b7761dc7141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:16.500055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-87c3fee6-1679-4bee-8e99-3b7761dc7141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:16.609287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3fee6-1679-4bee-8e99-3b7761dc7141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:16.610715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3fee6-1679-4bee-8e99-3b7761dc7141 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:16.629116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 733/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:18.573949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fc0d8-2b55-44f3-baa6-ad96d9fdb9c7 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:18.574384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fc0d8-2b55-44f3-baa6-ad96d9fdb9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:18.574493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fc0d8-2b55-44f3-baa6-ad96d9fdb9c7 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:18.576365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:18.582281 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cacb4-68b0-4066-b1a3-15c7b94e9ffc None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:18.582731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cacb4-68b0-4066-b1a3-15c7b94e9ffc None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:18.582839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cacb4-68b0-4066-b1a3-15c7b94e9ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:18.651229 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-581cacb4-68b0-4066-b1a3-15c7b94e9ffc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:18.652423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-581cacb4-68b0-4066-b1a3-15c7b94e9ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:18.702063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:18.750896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:19.035659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fc0d8-2b55-44f3-baa6-ad96d9fdb9c7 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:19.037351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-825fc0d8-2b55-44f3-baa6-ad96d9fdb9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:19.145309 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fc0d8-2b55-44f3-baa6-ad96d9fdb9c7 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:19.194296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:19.221127 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f29d74-25b0-43ce-abb0-cef8a2cb0308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:19.221730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f29d74-25b0-43ce-abb0-cef8a2cb0308 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:19.222065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f29d74-25b0-43ce-abb0-cef8a2cb0308 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:19.222346 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f29d74-25b0-43ce-abb0-cef8a2cb0308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:19.223951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a2f29d74-25b0-43ce-abb0-cef8a2cb0308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:19.336244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f29d74-25b0-43ce-abb0-cef8a2cb0308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:19.337785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f29d74-25b0-43ce-abb0-cef8a2cb0308 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:19.356237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 736/1470] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:28.749032 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=22 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:29.353951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=66 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:39.879805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed11d5-623e-483c-994a-f4ff2bd0fdd5 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:39.880344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed11d5-623e-483c-994a-f4ff2bd0fdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:39.880344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed11d5-623e-483c-994a-f4ff2bd0fdd5 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:39.881771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 735/1471] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:39.887497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234434b9-2a86-4b1e-9619-1135048e9724 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:39.887890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234434b9-2a86-4b1e-9619-1135048e9724 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:39.888236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234434b9-2a86-4b1e-9619-1135048e9724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:39.956875 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-234434b9-2a86-4b1e-9619-1135048e9724 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:39.958127 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-234434b9-2a86-4b1e-9619-1135048e9724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:40.058288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 737/1472] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:40.360832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c5ac6-79df-4c1b-b2f2-369e9cab2581 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:40.361231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c5ac6-79df-4c1b-b2f2-369e9cab2581 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:40.365359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c5ac6-79df-4c1b-b2f2-369e9cab2581 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:40.365359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 736/1473] 10.222.0.30 () {54 vars in 967 bytes} [Wed Aug 24 10:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:40.367740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b7f95-61a0-4f1b-9fa4-87d9c5d15d9a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:40.368029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b7f95-61a0-4f1b-9fa4-87d9c5d15d9a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:40.368308 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b7f95-61a0-4f1b-9fa4-87d9c5d15d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:40.447472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3f7b7f95-61a0-4f1b-9fa4-87d9c5d15d9a None None] MFA Rules not processed for user `d0b124ca267b4a4391557ad1396524cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:40.448744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3f7b7f95-61a0-4f1b-9fa4-87d9c5d15d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:40.471555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:40.536310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 738/1474] 10.222.0.30 () {58 vars in 1064 bytes} [Wed Aug 24 10:11:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:40.544166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c5ac6-79df-4c1b-b2f2-369e9cab2581 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:40.545819 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8f7c5ac6-79df-4c1b-b2f2-369e9cab2581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:40.656712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c5ac6-79df-4c1b-b2f2-369e9cab2581 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:40.734727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbb45b-aa72-4f11-b41a-faab49d61b21 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0b124ca267b4a4391557ad1396524cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:40.735332 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbb45b-aa72-4f11-b41a-faab49d61b21 service glance] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:40.735614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbb45b-aa72-4f11-b41a-faab49d61b21 service glance] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:40.735889 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbb45b-aa72-4f11-b41a-faab49d61b21 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:40.737652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-22cbb45b-aa72-4f11-b41a-faab49d61b21 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:40.793099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:40.848280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbb45b-aa72-4f11-b41a-faab49d61b21 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:40.850114 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbb45b-aa72-4f11-b41a-faab49d61b21 service glance] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:40.869601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 737/1475] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:11:42.896851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d27479-dce3-498a-9548-5e4fc0b6a2cf None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:42.897436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d27479-dce3-498a-9548-5e4fc0b6a2cf None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:42.897436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d27479-dce3-498a-9548-5e4fc0b6a2cf None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:42.899060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 739/1476] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:42.905087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3e684-f715-42c4-912b-3236ce347c3b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:42.905567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3e684-f715-42c4-912b-3236ce347c3b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:42.905678 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3e684-f715-42c4-912b-3236ce347c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:42.970802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-faa3e684-f715-42c4-912b-3236ce347c3b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:42.972017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-faa3e684-f715-42c4-912b-3236ce347c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:43.067771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 738/1477] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:43.343897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d27479-dce3-498a-9548-5e4fc0b6a2cf None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:43.345458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-33d27479-dce3-498a-9548-5e4fc0b6a2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:43.473824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d27479-dce3-498a-9548-5e4fc0b6a2cf None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:43.523654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:43.569052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45524c68-4f2d-4160-b175-3f0eecd2bf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:43.569608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45524c68-4f2d-4160-b175-3f0eecd2bf39 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:43.569883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45524c68-4f2d-4160-b175-3f0eecd2bf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:43.570149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45524c68-4f2d-4160-b175-3f0eecd2bf39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:43.570777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45524c68-4f2d-4160-b175-3f0eecd2bf39 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:43.572539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: WARNING py.warnings [None req-45524c68-4f2d-4160-b175-3f0eecd2bf39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:11:43.572539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: warnings.warn(msg) Aug 24 10:11:43.572539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: Aug 24 10:11:43.573698 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45524c68-4f2d-4160-b175-3f0eecd2bf39 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:43.715683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 740/1478] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Aug 24 10:11:43 2022] GET /identity/v3/projects => generated 2553 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:11:45.646140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf7df1-225a-4b77-92fa-56b32dd93dae None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:45.646705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf7df1-225a-4b77-92fa-56b32dd93dae None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:45.646705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf7df1-225a-4b77-92fa-56b32dd93dae None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:45.648056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 739/1479] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:45.654732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1398b9c-5500-4e45-a8f9-de2388587aef None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:45.655171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1398b9c-5500-4e45-a8f9-de2388587aef None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:45.655348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1398b9c-5500-4e45-a8f9-de2388587aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:45.722867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-f1398b9c-5500-4e45-a8f9-de2388587aef None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:45.725589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f1398b9c-5500-4e45-a8f9-de2388587aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:45.822622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 741/1480] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:46.124038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf7df1-225a-4b77-92fa-56b32dd93dae None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:46.125620 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-aecf7df1-225a-4b77-92fa-56b32dd93dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:46.135729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:46.274249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf7df1-225a-4b77-92fa-56b32dd93dae None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:46.354975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:46.388242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52572b0-f86d-473a-8d13-696601ef21cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:46.389267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52572b0-f86d-473a-8d13-696601ef21cb service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:46.389550 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52572b0-f86d-473a-8d13-696601ef21cb service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:46.389824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52572b0-f86d-473a-8d13-696601ef21cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:46.391423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d52572b0-f86d-473a-8d13-696601ef21cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:46.484233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52572b0-f86d-473a-8d13-696601ef21cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:46.485878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52572b0-f86d-473a-8d13-696601ef21cb service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:46.503792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 740/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:48.635101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c762d-d108-4997-822f-9fd8cfc52e3a None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:48.635718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c762d-d108-4997-822f-9fd8cfc52e3a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:48.635718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c762d-d108-4997-822f-9fd8cfc52e3a None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:48.637846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 742/1482] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:48.643323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba1f72-cf12-46ed-b237-25f8c195f9ab None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:48.643775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba1f72-cf12-46ed-b237-25f8c195f9ab None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:48.643880 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba1f72-cf12-46ed-b237-25f8c195f9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:48.709348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-eeba1f72-cf12-46ed-b237-25f8c195f9ab None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:48.710559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-eeba1f72-cf12-46ed-b237-25f8c195f9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:48.757425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:48.808181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 741/1483] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:49.093243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c762d-d108-4997-822f-9fd8cfc52e3a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:49.094817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a44c762d-d108-4997-822f-9fd8cfc52e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:49.118908 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:49.201475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c762d-d108-4997-822f-9fd8cfc52e3a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:49.288951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f4603-59d3-44cb-84e3-b67cce848917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:49.289566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f4603-59d3-44cb-84e3-b67cce848917 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:49.289848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f4603-59d3-44cb-84e3-b67cce848917 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:49.290123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f4603-59d3-44cb-84e3-b67cce848917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:49.291753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2b3f4603-59d3-44cb-84e3-b67cce848917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:49.369766 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:49.405363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f4603-59d3-44cb-84e3-b67cce848917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:49.406924 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f4603-59d3-44cb-84e3-b67cce848917 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:49.426611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 743/1484] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:51.427284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2154-03d6-4160-9a0d-262c5b76976a None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:51.427877 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2154-03d6-4160-9a0d-262c5b76976a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:51.427877 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2154-03d6-4160-9a0d-262c5b76976a None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:51.429236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 742/1485] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:51.435530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd9401-2456-45ac-8e5b-8861d2b3ba9a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:51.436071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd9401-2456-45ac-8e5b-8861d2b3ba9a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:51.436071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd9401-2456-45ac-8e5b-8861d2b3ba9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:51.504939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-6dfd9401-2456-45ac-8e5b-8861d2b3ba9a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:51.506190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6dfd9401-2456-45ac-8e5b-8861d2b3ba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:51.607929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 744/1486] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:11:51.905623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd2154-03d6-4160-9a0d-262c5b76976a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:51.907386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d7cd2154-03d6-4160-9a0d-262c5b76976a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:52.015072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd2154-03d6-4160-9a0d-262c5b76976a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:52.085608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:52.087359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe9826-4fd6-4179-9ebf-e4ff0ccc5efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:52.087961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe9826-4fd6-4179-9ebf-e4ff0ccc5efb service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:52.088319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe9826-4fd6-4179-9ebf-e4ff0ccc5efb service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:52.088596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe9826-4fd6-4179-9ebf-e4ff0ccc5efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:52.090867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-24fe9826-4fd6-4179-9ebf-e4ff0ccc5efb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:52.197222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe9826-4fd6-4179-9ebf-e4ff0ccc5efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:52.198573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe9826-4fd6-4179-9ebf-e4ff0ccc5efb service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:52.217383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 743/1487] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:11:54.266632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f6fe9-80ca-47b7-a52d-a7ab0fadeb02 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:54.267192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f6fe9-80ca-47b7-a52d-a7ab0fadeb02 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:54.267192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f6fe9-80ca-47b7-a52d-a7ab0fadeb02 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:54.268568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 745/1488] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:54.274385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c1fe0-4d89-41ee-a314-9780535f3699 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:54.274861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c1fe0-4d89-41ee-a314-9780535f3699 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:54.274956 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c1fe0-4d89-41ee-a314-9780535f3699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:54.345579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-953c1fe0-4d89-41ee-a314-9780535f3699 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:54.346794 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-953c1fe0-4d89-41ee-a314-9780535f3699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:54.399125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:54.446558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 744/1489] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:54.743214 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f6fe9-80ca-47b7-a52d-a7ab0fadeb02 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:54.744743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2c5f6fe9-80ca-47b7-a52d-a7ab0fadeb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:54.781320 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:54.849460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f6fe9-80ca-47b7-a52d-a7ab0fadeb02 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:54.929273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368fbeb-b1a2-4f3c-ab24-8bc320a8f000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:54.929845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368fbeb-b1a2-4f3c-ab24-8bc320a8f000 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:54.930124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368fbeb-b1a2-4f3c-ab24-8bc320a8f000 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:54.930394 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368fbeb-b1a2-4f3c-ab24-8bc320a8f000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:54.936611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c368fbeb-b1a2-4f3c-ab24-8bc320a8f000 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:55.024357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:55.053777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368fbeb-b1a2-4f3c-ab24-8bc320a8f000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:55.055205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368fbeb-b1a2-4f3c-ab24-8bc320a8f000 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:55.074587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 746/1490] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:57.451802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354ebbb-3986-4a67-befc-c317e2e392f4 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:57.452373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354ebbb-3986-4a67-befc-c317e2e392f4 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:57.452373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354ebbb-3986-4a67-befc-c317e2e392f4 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:57.453911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 745/1491] 10.222.0.30 () {54 vars in 927 bytes} [Wed Aug 24 10:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:57.459444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc627b-318e-4ff3-9e33-15c5af14d607 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:57.459915 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc627b-318e-4ff3-9e33-15c5af14d607 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:57.460021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc627b-318e-4ff3-9e33-15c5af14d607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:57.532243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-68dc627b-318e-4ff3-9e33-15c5af14d607 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:57.533493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-68dc627b-318e-4ff3-9e33-15c5af14d607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:57.636221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 747/1492] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Aug 24 10:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:57.940525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354ebbb-3986-4a67-befc-c317e2e392f4 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:57.942209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e354ebbb-3986-4a67-befc-c317e2e392f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:58.057116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354ebbb-3986-4a67-befc-c317e2e392f4 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:58.135290 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bba565-886f-4c59-ba99-d58369291c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:58.135891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bba565-886f-4c59-ba99-d58369291c07 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:58.136838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bba565-886f-4c59-ba99-d58369291c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:58.137211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bba565-886f-4c59-ba99-d58369291c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:58.138899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f7bba565-886f-4c59-ba99-d58369291c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:58.149718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:58.261975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bba565-886f-4c59-ba99-d58369291c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:58.263359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bba565-886f-4c59-ba99-d58369291c07 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:58.283787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 746/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:12:07.634276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=82 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:08.281373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=50 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:20.409642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71974f7f-b4a6-4724-b08c-fd981e5912cd None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:20.410256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71974f7f-b4a6-4724-b08c-fd981e5912cd None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:20.410256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71974f7f-b4a6-4724-b08c-fd981e5912cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:20.495651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-71974f7f-b4a6-4724-b08c-fd981e5912cd None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:20.496895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-71974f7f-b4a6-4724-b08c-fd981e5912cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:20.599871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 748/1494] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:20.609739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497001d-539b-4923-a561-b9cb58fcc127 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:20.611324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497001d-539b-4923-a561-b9cb58fcc127 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:20.611324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497001d-539b-4923-a561-b9cb58fcc127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:20.691170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-7497001d-539b-4923-a561-b9cb58fcc127 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:20.692383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7497001d-539b-4923-a561-b9cb58fcc127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:20.789792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 747/1495] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:20.794485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71974f7f-b4a6-4724-b08c-fd981e5912cd None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:20.795690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-71974f7f-b4a6-4724-b08c-fd981e5912cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:20.808181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71974f7f-b4a6-4724-b08c-fd981e5912cd None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:20.875122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:20.910216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f9f05-ba00-4872-af6d-4c3a1d304c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:20.910828 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f9f05-ba00-4872-af6d-4c3a1d304c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:20.911106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f9f05-ba00-4872-af6d-4c3a1d304c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:20.911381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f9f05-ba00-4872-af6d-4c3a1d304c71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:20.912023 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f9f05-ba00-4872-af6d-4c3a1d304c71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:13:20.912348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f9f05-ba00-4872-af6d-4c3a1d304c71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:20.914015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: WARNING py.warnings [None req-4a8f9f05-ba00-4872-af6d-4c3a1d304c71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:13:20.914015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: warnings.warn(msg) Aug 24 10:13:20.914015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: Aug 24 10:13:20.914905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f9f05-ba00-4872-af6d-4c3a1d304c71 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:20.926273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 749/1496] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:13:20.935210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497001d-539b-4923-a561-b9cb58fcc127 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:20.944933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497001d-539b-4923-a561-b9cb58fcc127 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:21.006902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:21.040982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c6ba0-c2e5-4c71-a7dd-86be0be33947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:21.041620 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c6ba0-c2e5-4c71-a7dd-86be0be33947 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:21.041926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c6ba0-c2e5-4c71-a7dd-86be0be33947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:21.042202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c6ba0-c2e5-4c71-a7dd-86be0be33947 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:21.042837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c6ba0-c2e5-4c71-a7dd-86be0be33947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:21.044566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: WARNING py.warnings [None req-0f6c6ba0-c2e5-4c71-a7dd-86be0be33947 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:13:21.044566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: warnings.warn(msg) Aug 24 10:13:21.044566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: Aug 24 10:13:21.045839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c6ba0-c2e5-4c71-a7dd-86be0be33947 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:21.247647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 748/1497] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:13:20 2022] POST /identity/v3/projects => generated 377 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:13:21.251820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f9f05-ba00-4872-af6d-4c3a1d304c71 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:21.253401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4a8f9f05-ba00-4872-af6d-4c3a1d304c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:21.288688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f9f05-ba00-4872-af6d-4c3a1d304c71 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:21.384539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388782b-331c-4e42-b89b-afce2f193fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:21.385175 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388782b-331c-4e42-b89b-afce2f193fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:21.385457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388782b-331c-4e42-b89b-afce2f193fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:21.385729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388782b-331c-4e42-b89b-afce2f193fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:21.386363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388782b-331c-4e42-b89b-afce2f193fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:21.387365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: WARNING py.warnings [None req-9388782b-331c-4e42-b89b-afce2f193fe6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:13:21.387365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: warnings.warn(msg) Aug 24 10:13:21.387365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: Aug 24 10:13:21.388568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388782b-331c-4e42-b89b-afce2f193fe6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:21.516224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 750/1498] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:13:21 2022] POST /identity/v3/users => generated 289 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:13:21.520263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c6ba0-c2e5-4c71-a7dd-86be0be33947 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:21.521872 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0f6c6ba0-c2e5-4c71-a7dd-86be0be33947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:21.587034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c6ba0-c2e5-4c71-a7dd-86be0be33947 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:21.669612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:21.680068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aeadd5-d149-4ddc-bdbc-6768539a3082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:21.680669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeadd5-d149-4ddc-bdbc-6768539a3082 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:21.680977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeadd5-d149-4ddc-bdbc-6768539a3082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:21.681273 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeadd5-d149-4ddc-bdbc-6768539a3082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:21.681951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aeadd5-d149-4ddc-bdbc-6768539a3082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:21.682952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: WARNING py.warnings [None req-e2aeadd5-d149-4ddc-bdbc-6768539a3082 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:13:21.682952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: warnings.warn(msg) Aug 24 10:13:21.682952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: Aug 24 10:13:21.683833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aeadd5-d149-4ddc-bdbc-6768539a3082 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:21.697019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 749/1499] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:13:21.700851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388782b-331c-4e42-b89b-afce2f193fe6 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:21.702431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9388782b-331c-4e42-b89b-afce2f193fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:21.714568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388782b-331c-4e42-b89b-afce2f193fe6 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:21.740362 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:21.810769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03750b16-1956-41de-aade-497c26e23b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:21.811294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03750b16-1956-41de-aade-497c26e23b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:21.811573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03750b16-1956-41de-aade-497c26e23b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:21.811843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03750b16-1956-41de-aade-497c26e23b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:21.812484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03750b16-1956-41de-aade-497c26e23b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:21.813526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: WARNING py.warnings [None req-03750b16-1956-41de-aade-497c26e23b35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:13:21.813526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: warnings.warn(msg) Aug 24 10:13:21.813526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: Aug 24 10:13:21.814445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03750b16-1956-41de-aade-497c26e23b35 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:21.827636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 751/1500] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:13:21.832078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aeadd5-d149-4ddc-bdbc-6768539a3082 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:21.833659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e2aeadd5-d149-4ddc-bdbc-6768539a3082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:21.867542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aeadd5-d149-4ddc-bdbc-6768539a3082 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:21.962396 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54107677-de46-4198-afe2-d8011cd7e464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:21.963051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54107677-de46-4198-afe2-d8011cd7e464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:21.963334 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54107677-de46-4198-afe2-d8011cd7e464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:21.963612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54107677-de46-4198-afe2-d8011cd7e464 admin admin] PATH_INFO: `/v3/projects/f79d91cb685a4933aabbd0566f137c1a/users/40e7359729184c33974b58866a807a18/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:21.985208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54107677-de46-4198-afe2-d8011cd7e464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f79d91cb685a4933aabbd0566f137c1a, user_id=40e7359729184c33974b58866a807a18, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:21.986262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: WARNING py.warnings [None req-54107677-de46-4198-afe2-d8011cd7e464 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:13:21.986262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: warnings.warn(msg) Aug 24 10:13:21.986262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: Aug 24 10:13:21.988202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54107677-de46-4198-afe2-d8011cd7e464 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:22.074893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 750/1501] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:13:21 2022] PUT /identity/v3/projects/f79d91cb685a4933aabbd0566f137c1a/users/40e7359729184c33974b58866a807a18/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:13:22.087590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183d5c9-a7e5-44c6-846d-f09c3d6aac3e None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:22.087875 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183d5c9-a7e5-44c6-846d-f09c3d6aac3e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:22.088257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183d5c9-a7e5-44c6-846d-f09c3d6aac3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:22.151358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0183d5c9-a7e5-44c6-846d-f09c3d6aac3e None None] MFA Rules not processed for user `40e7359729184c33974b58866a807a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:22.152663 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0183d5c9-a7e5-44c6-846d-f09c3d6aac3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:22.230125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 752/1502] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:13:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:22.244286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d48205-bb5c-415f-a246-c90ca68794c6 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:22.244635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d48205-bb5c-415f-a246-c90ca68794c6 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:22.245024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d48205-bb5c-415f-a246-c90ca68794c6 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:22.246412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 751/1503] 10.222.0.30 () {54 vars in 964 bytes} [Wed Aug 24 10:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:13:22.251449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaa15b-3714-42e9-a8d8-4d29259c9445 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:22.251761 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaa15b-3714-42e9-a8d8-4d29259c9445 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:22.252035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaa15b-3714-42e9-a8d8-4d29259c9445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:22.283245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:22.329869 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-6ceaa15b-3714-42e9-a8d8-4d29259c9445 None None] MFA Rules not processed for user `c71783d1327a47149a74414218dd7219`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:22.331013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6ceaa15b-3714-42e9-a8d8-4d29259c9445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:22.407798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 753/1504] 10.222.0.30 () {58 vars in 1061 bytes} [Wed Aug 24 10:13:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:13:22.416707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d48205-bb5c-415f-a246-c90ca68794c6 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:22.418318 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-32d48205-bb5c-415f-a246-c90ca68794c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:22.469839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:22.524209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d48205-bb5c-415f-a246-c90ca68794c6 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:22.596914 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd078190-900d-4af7-9928-695700b94f69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:22.597444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd078190-900d-4af7-9928-695700b94f69 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:22.597936 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd078190-900d-4af7-9928-695700b94f69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:22.598254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd078190-900d-4af7-9928-695700b94f69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:22.601158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-cd078190-900d-4af7-9928-695700b94f69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:22.706634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd078190-900d-4af7-9928-695700b94f69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:22.708061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd078190-900d-4af7-9928-695700b94f69 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:22.712404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:22.727466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 752/1505] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:13:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:13:22.777823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684dc60-d54e-4494-bab9-e1c372034586 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:22.778321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684dc60-d54e-4494-bab9-e1c372034586 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:22.778431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684dc60-d54e-4494-bab9-e1c372034586 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:22.779650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 754/1506] 10.222.0.30 () {52 vars in 830 bytes} [Wed Aug 24 10:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:13:22.783501 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd078190-900d-4af7-9928-695700b94f69 service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:22.784715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-cd078190-900d-4af7-9928-695700b94f69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:22.893332 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd078190-900d-4af7-9928-695700b94f69 service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:23.005655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:23.006296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:23.007106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:23.007378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] PATH_INFO: `/v3/users/40e7359729184c33974b58866a807a18` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:23.023069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e7359729184c33974b58866a807a18)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:23.024823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: WARNING py.warnings [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:13:23.024823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: warnings.warn(msg) Aug 24 10:13:23.024823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: Aug 24 10:13:23.026103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:23.036471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:23.303587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e7359729184c33974b58866a807a18'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:23.304447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e7359729184c33974b58866a807a18'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:23.320263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e7359729184c33974b58866a807a18'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:23.320733 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e7359729184c33974b58866a807a18'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:23.432508 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e7359729184c33974b58866a807a18'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:23.467048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 753/1507] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:13:22 2022] DELETE /identity/v3/users/40e7359729184c33974b58866a807a18 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:23.471805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ad734-3c5f-4e0e-a886-6e908092a898 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:23.472119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ad734-3c5f-4e0e-a886-6e908092a898 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:23.472428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ad734-3c5f-4e0e-a886-6e908092a898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:23.534107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-927ad734-3c5f-4e0e-a886-6e908092a898 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:23.535361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-927ad734-3c5f-4e0e-a886-6e908092a898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:23.617177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:23.630835 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 755/1508] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:13:23.642663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:23.644073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:23.767150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b2764-98b7-4697-a399-06d88f9f462a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:23.830997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:23.855824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd381b9-d70e-4860-bed1-1e3bc3bce795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:23.856443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd381b9-d70e-4860-bed1-1e3bc3bce795 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:23.856731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd381b9-d70e-4860-bed1-1e3bc3bce795 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:23.857028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd381b9-d70e-4860-bed1-1e3bc3bce795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:23.858666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-4cd381b9-d70e-4860-bed1-1e3bc3bce795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:23.986137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd381b9-d70e-4860-bed1-1e3bc3bce795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:23.987499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd381b9-d70e-4860-bed1-1e3bc3bce795 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:24.008656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 754/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:13:24.521834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ad734-3c5f-4e0e-a886-6e908092a898 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:24.523715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-927ad734-3c5f-4e0e-a886-6e908092a898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:24.536851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ad734-3c5f-4e0e-a886-6e908092a898 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:24.645388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:24.646488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:24.646846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:24.647835 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] PATH_INFO: `/v3/projects/f79d91cb685a4933aabbd0566f137c1a` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:24.658911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f79d91cb685a4933aabbd0566f137c1a)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:24.659949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: WARNING py.warnings [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:13:24.659949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: warnings.warn(msg) Aug 24 10:13:24.659949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: Aug 24 10:13:24.661220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:24.795483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f79d91cb685a4933aabbd0566f137c1a', 'request_id': 'req-59177ed5-7d81-4a58-b9fa-225da02090ae'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:24.802155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f79d91cb685a4933aabbd0566f137c1a', 'request_id': 'req-59177ed5-7d81-4a58-b9fa-225da02090ae'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:24.885340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f79d91cb685a4933aabbd0566f137c1a', 'request_id': 'req-59177ed5-7d81-4a58-b9fa-225da02090ae'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:24.916810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] The token cache is being invalidate because project f79d91cb685a4933aabbd0566f137c1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:13:24.917130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:24.917689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59177ed5-7d81-4a58-b9fa-225da02090ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:24.919008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 756/1510] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:13:24 2022] DELETE /identity/v3/projects/f79d91cb685a4933aabbd0566f137c1a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:34.028967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=62 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:34.914748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:54.988161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c23224-b6f9-4bc2-8fe1-97f53c97f3e9 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:54.988741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c23224-b6f9-4bc2-8fe1-97f53c97f3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:54.988741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c23224-b6f9-4bc2-8fe1-97f53c97f3e9 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:54.990066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:14:54.995034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc389b66-2b26-4045-96e6-552db2aafa7c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:54.998823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc389b66-2b26-4045-96e6-552db2aafa7c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:54.998823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc389b66-2b26-4045-96e6-552db2aafa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:55.080591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-cc389b66-2b26-4045-96e6-552db2aafa7c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:14:55.081826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cc389b66-2b26-4045-96e6-552db2aafa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:55.183740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:14:55.341577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c23224-b6f9-4bc2-8fe1-97f53c97f3e9 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:55.343822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b3c23224-b6f9-4bc2-8fe1-97f53c97f3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:55.465278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c23224-b6f9-4bc2-8fe1-97f53c97f3e9 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:55.470873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d277e4-fa38-4597-af1b-c15729baae24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:55.471477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d277e4-fa38-4597-af1b-c15729baae24 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:55.471762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d277e4-fa38-4597-af1b-c15729baae24 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:55.472039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d277e4-fa38-4597-af1b-c15729baae24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:55.480217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d277e4-fa38-4597-af1b-c15729baae24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:55.481858 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d277e4-fa38-4597-af1b-c15729baae24 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:55.492400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 756/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:14:56.349200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34854d45-313d-4f7f-b371-06ede3d49a66 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:56.349763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34854d45-313d-4f7f-b371-06ede3d49a66 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:56.349763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34854d45-313d-4f7f-b371-06ede3d49a66 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:56.351107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:14:56.355930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cb5d5-6a9d-49f6-a76e-0b551b84cb8e None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:56.356355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cb5d5-6a9d-49f6-a76e-0b551b84cb8e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:56.356495 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cb5d5-6a9d-49f6-a76e-0b551b84cb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:56.376790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-374cb5d5-6a9d-49f6-a76e-0b551b84cb8e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:14:56.378168 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-374cb5d5-6a9d-49f6-a76e-0b551b84cb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:56.386343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:14:56.538108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34854d45-313d-4f7f-b371-06ede3d49a66 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:56.552731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34854d45-313d-4f7f-b371-06ede3d49a66 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:56.558275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1e08c-553f-446c-8280-e1d8026488e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:56.558902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1e08c-553f-446c-8280-e1d8026488e1 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:56.559224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1e08c-553f-446c-8280-e1d8026488e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:56.559484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1e08c-553f-446c-8280-e1d8026488e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:56.569656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1e08c-553f-446c-8280-e1d8026488e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:56.571277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1e08c-553f-446c-8280-e1d8026488e1 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:56.581690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 759/1516] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:14:56.766881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41eea3-6b73-4b7d-8f8a-5d499d778951 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:56.767221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41eea3-6b73-4b7d-8f8a-5d499d778951 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:56.767813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41eea3-6b73-4b7d-8f8a-5d499d778951 None None] PATH_INFO: `/v3` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:56.769232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 758/1517] 10.222.0.81 () {54 vars in 925 bytes} [Wed Aug 24 10:14:56 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:14:56.775748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df979e0f-882a-4afe-88b2-7ee2e1d7ed7a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:56.776061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df979e0f-882a-4afe-88b2-7ee2e1d7ed7a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:56.776333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df979e0f-882a-4afe-88b2-7ee2e1d7ed7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:56.837431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-df979e0f-882a-4afe-88b2-7ee2e1d7ed7a None None] MFA Rules not processed for user `b1aa47a883604336a8ab90d324470477`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:14:56.838579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-df979e0f-882a-4afe-88b2-7ee2e1d7ed7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:56.855850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:56.916058 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 760/1518] 10.222.0.81 () {58 vars in 1015 bytes} [Wed Aug 24 10:14:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:15:06.067310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cced73-a733-49f6-adbd-9b91556d05b4 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:06.067843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cced73-a733-49f6-adbd-9b91556d05b4 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:06.067843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cced73-a733-49f6-adbd-9b91556d05b4 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:06.069143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 759/1519] 10.222.0.30 () {54 vars in 961 bytes} [Wed Aug 24 10:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:15:06.074264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839fe72-99a4-4dd8-9e16-3bd020db889a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:06.074700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839fe72-99a4-4dd8-9e16-3bd020db889a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:06.074806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839fe72-99a4-4dd8-9e16-3bd020db889a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:06.096499 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-c839fe72-99a4-4dd8-9e16-3bd020db889a None None] MFA Rules not processed for user `b1aa47a883604336a8ab90d324470477`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:15:06.097790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c839fe72-99a4-4dd8-9e16-3bd020db889a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:06.106795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 761/1520] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Aug 24 10:15:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:15:06.115380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cced73-a733-49f6-adbd-9b91556d05b4 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:06.130817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cced73-a733-49f6-adbd-9b91556d05b4 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:06.135738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57799e-050f-4581-860a-490f5e8a93ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:06.136343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57799e-050f-4581-860a-490f5e8a93ec service placement] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:06.136622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57799e-050f-4581-860a-490f5e8a93ec service placement] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:06.136926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57799e-050f-4581-860a-490f5e8a93ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:06.146042 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57799e-050f-4581-860a-490f5e8a93ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:06.147600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57799e-050f-4581-860a-490f5e8a93ec service placement] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:06.157270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 760/1521] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:07.836810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caed35-ff26-42ec-9f7b-278b35bab018 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:07.837417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caed35-ff26-42ec-9f7b-278b35bab018 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:07.837417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caed35-ff26-42ec-9f7b-278b35bab018 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:07.838749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:15:07.843760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fafa86-2efd-46fc-a2f3-30c294f3fffc None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:07.844142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fafa86-2efd-46fc-a2f3-30c294f3fffc None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:07.844892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fafa86-2efd-46fc-a2f3-30c294f3fffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:07.864793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-05fafa86-2efd-46fc-a2f3-30c294f3fffc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:15:07.866026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-05fafa86-2efd-46fc-a2f3-30c294f3fffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:07.874433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:15:08.035524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4caed35-ff26-42ec-9f7b-278b35bab018 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:08.052093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4caed35-ff26-42ec-9f7b-278b35bab018 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:08.058172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c686014-046c-41c4-b9f0-9815db1a4be7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:08.058818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c686014-046c-41c4-b9f0-9815db1a4be7 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:08.059102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c686014-046c-41c4-b9f0-9815db1a4be7 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:08.059377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c686014-046c-41c4-b9f0-9815db1a4be7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:08.069097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c686014-046c-41c4-b9f0-9815db1a4be7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:08.070690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c686014-046c-41c4-b9f0-9815db1a4be7 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:08.080179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 763/1524] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:08.932543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3a978-7d76-4aa1-99a7-a8e789181a8d None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:08.933151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3a978-7d76-4aa1-99a7-a8e789181a8d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:08.933151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3a978-7d76-4aa1-99a7-a8e789181a8d None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:08.934599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:15:08.940515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a104a3a-9b84-452d-bb5f-3306f28cfe25 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:08.941033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a104a3a-9b84-452d-bb5f-3306f28cfe25 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:08.941132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a104a3a-9b84-452d-bb5f-3306f28cfe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:08.960418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-2a104a3a-9b84-452d-bb5f-3306f28cfe25 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:15:08.961668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2a104a3a-9b84-452d-bb5f-3306f28cfe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:08.969992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:15:09.124531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3a978-7d76-4aa1-99a7-a8e789181a8d None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:09.139905 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3a978-7d76-4aa1-99a7-a8e789181a8d None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:09.145787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc31e6-1e3a-488e-bd4e-c03184e843f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:09.146389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc31e6-1e3a-488e-bd4e-c03184e843f4 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:09.146666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc31e6-1e3a-488e-bd4e-c03184e843f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:09.146939 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc31e6-1e3a-488e-bd4e-c03184e843f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:09.155178 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc31e6-1e3a-488e-bd4e-c03184e843f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:09.157304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc31e6-1e3a-488e-bd4e-c03184e843f4 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:09.166494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 763/1527] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:13.202671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87554e8b-8abf-42fe-b3fa-64ddda99f475 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:13.203256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87554e8b-8abf-42fe-b3fa-64ddda99f475 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:13.203256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87554e8b-8abf-42fe-b3fa-64ddda99f475 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:13.204581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:15:13.210906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb78d8-796c-4b8a-85e9-1ebac2e094a7 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:13.211329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb78d8-796c-4b8a-85e9-1ebac2e094a7 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:13.211482 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb78d8-796c-4b8a-85e9-1ebac2e094a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:13.230893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e7cb78d8-796c-4b8a-85e9-1ebac2e094a7 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:15:13.232097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e7cb78d8-796c-4b8a-85e9-1ebac2e094a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:13.240603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:15:13.549498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87554e8b-8abf-42fe-b3fa-64ddda99f475 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:13.551351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-87554e8b-8abf-42fe-b3fa-64ddda99f475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:13.648745 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87554e8b-8abf-42fe-b3fa-64ddda99f475 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:13.653757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebaa3c-7aba-48bf-b2b6-487549f64c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:13.654517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebaa3c-7aba-48bf-b2b6-487549f64c32 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:13.654814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebaa3c-7aba-48bf-b2b6-487549f64c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:13.655061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebaa3c-7aba-48bf-b2b6-487549f64c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:13.666589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebaa3c-7aba-48bf-b2b6-487549f64c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:13.668049 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebaa3c-7aba-48bf-b2b6-487549f64c32 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:13.680993 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 766/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:15.648438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73fe52-f4e5-4d19-a8e2-a1283f47a841 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:15.649023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73fe52-f4e5-4d19-a8e2-a1283f47a841 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:15.649023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73fe52-f4e5-4d19-a8e2-a1283f47a841 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:15.650365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:15:15.655548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f16d-006d-4ac2-af51-5369064198ec None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:15.655974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f16d-006d-4ac2-af51-5369064198ec None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:15.656124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f16d-006d-4ac2-af51-5369064198ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:15.677796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-7536f16d-006d-4ac2-af51-5369064198ec None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:15:15.679031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7536f16d-006d-4ac2-af51-5369064198ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:15.687107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:15:15.973041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73fe52-f4e5-4d19-a8e2-a1283f47a841 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:15.983802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73fe52-f4e5-4d19-a8e2-a1283f47a841 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:15.988304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a954d3-ec29-464a-a1a7-df7b84eccec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:15.988922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a954d3-ec29-464a-a1a7-df7b84eccec5 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:15.989203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a954d3-ec29-464a-a1a7-df7b84eccec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:15.989474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a954d3-ec29-464a-a1a7-df7b84eccec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:15.998635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a954d3-ec29-464a-a1a7-df7b84eccec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:16.000376 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a954d3-ec29-464a-a1a7-df7b84eccec5 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:16.009786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 766/1533] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:19.475225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3671ad-7462-4540-a15d-095f8be9e190 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:19.475768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3671ad-7462-4540-a15d-095f8be9e190 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:19.475902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3671ad-7462-4540-a15d-095f8be9e190 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:19.477410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:15:19.482517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72d245-a3d2-4590-82ce-08d43710edd8 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:19.482936 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72d245-a3d2-4590-82ce-08d43710edd8 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:19.483072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72d245-a3d2-4590-82ce-08d43710edd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:19.502710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6b72d245-a3d2-4590-82ce-08d43710edd8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:15:19.503904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6b72d245-a3d2-4590-82ce-08d43710edd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:19.511833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:15:20.076215 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3671ad-7462-4540-a15d-095f8be9e190 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:20.078703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5f3671ad-7462-4540-a15d-095f8be9e190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:20.145803 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:15:20.159302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3671ad-7462-4540-a15d-095f8be9e190 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:20.162929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfe39b-3360-4713-994e-719060fee7dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:20.163531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe39b-3360-4713-994e-719060fee7dc service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:20.163807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe39b-3360-4713-994e-719060fee7dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:20.164076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe39b-3360-4713-994e-719060fee7dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:20.173722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfe39b-3360-4713-994e-719060fee7dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:20.175144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfe39b-3360-4713-994e-719060fee7dc service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:20.185011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 769/1536] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:22.140735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68711f9-f7f9-4296-a2d4-cbc571206ecb None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:22.141443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68711f9-f7f9-4296-a2d4-cbc571206ecb None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:22.141561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68711f9-f7f9-4296-a2d4-cbc571206ecb None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:22.142872 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:15:22.148169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77286e9d-be19-4104-9258-e58fa5f9c3e5 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:22.148613 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77286e9d-be19-4104-9258-e58fa5f9c3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:22.148721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77286e9d-be19-4104-9258-e58fa5f9c3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:22.168410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-77286e9d-be19-4104-9258-e58fa5f9c3e5 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:15:22.169669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-77286e9d-be19-4104-9258-e58fa5f9c3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:22.178167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:15:22.739251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68711f9-f7f9-4296-a2d4-cbc571206ecb None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:22.740955 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e68711f9-f7f9-4296-a2d4-cbc571206ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:22.753989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68711f9-f7f9-4296-a2d4-cbc571206ecb None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:22.757496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd8d3c-d6a8-4660-98ff-23a017ccc827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:22.758079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd8d3c-d6a8-4660-98ff-23a017ccc827 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:22.758357 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd8d3c-d6a8-4660-98ff-23a017ccc827 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:22.758626 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd8d3c-d6a8-4660-98ff-23a017ccc827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:22.767497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd8d3c-d6a8-4660-98ff-23a017ccc827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:22.768935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd8d3c-d6a8-4660-98ff-23a017ccc827 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:22.778147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 769/1539] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:32.163689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=11 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:15:32.774092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:45.730247 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ddc72-192d-4449-806a-3026a7d501ea None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:45.737741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ddc72-192d-4449-806a-3026a7d501ea None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:45.737741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ddc72-192d-4449-806a-3026a7d501ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:45.761711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-149ddc72-192d-4449-806a-3026a7d501ea None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:45.762952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-149ddc72-192d-4449-806a-3026a7d501ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:45.774400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 771/1540] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:45.783995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a73966-4b1d-4595-94d6-27b775121328 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:45.784553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a73966-4b1d-4595-94d6-27b775121328 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:45.784646 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a73966-4b1d-4595-94d6-27b775121328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:45.818106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-64a73966-4b1d-4595-94d6-27b775121328 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:45.819762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-64a73966-4b1d-4595-94d6-27b775121328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:45.833271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 770/1541] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:45.838210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ddc72-192d-4449-806a-3026a7d501ea None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:45.852433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ddc72-192d-4449-806a-3026a7d501ea None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:45.861321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5745204-7eba-4e12-96c4-2d542909c7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:45.861967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5745204-7eba-4e12-96c4-2d542909c7ae admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:45.862244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5745204-7eba-4e12-96c4-2d542909c7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:45.862519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5745204-7eba-4e12-96c4-2d542909c7ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:45.863213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5745204-7eba-4e12-96c4-2d542909c7ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:16:45.863524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5745204-7eba-4e12-96c4-2d542909c7ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:45.865284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5745204-7eba-4e12-96c4-2d542909c7ae admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:45.889065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 772/1542] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:16:45.897707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a73966-4b1d-4595-94d6-27b775121328 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:45.912889 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a73966-4b1d-4595-94d6-27b775121328 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:45.925988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98799629-ae33-4895-b2e1-b25fd566418a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:45.927021 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98799629-ae33-4895-b2e1-b25fd566418a admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:45.927524 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98799629-ae33-4895-b2e1-b25fd566418a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:45.928015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98799629-ae33-4895-b2e1-b25fd566418a admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:45.928879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98799629-ae33-4895-b2e1-b25fd566418a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:45.931060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98799629-ae33-4895-b2e1-b25fd566418a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:45.957103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb29d4-7434-48c5-906f-ed886f2361b7 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:45.957460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb29d4-7434-48c5-906f-ed886f2361b7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:45.957652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb29d4-7434-48c5-906f-ed886f2361b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.001357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-9dbb29d4-7434-48c5-906f-ed886f2361b7 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.002521 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9dbb29d4-7434-48c5-906f-ed886f2361b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.015690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 773/1543] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.024368 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd033e-d16d-4af0-8244-144800d0a687 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.035205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd033e-d16d-4af0-8244-144800d0a687 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.035205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd033e-d16d-4af0-8244-144800d0a687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.057354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-f1fd033e-d16d-4af0-8244-144800d0a687 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.059573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f1fd033e-d16d-4af0-8244-144800d0a687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.069438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 771/1544] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:16:45 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:46.071768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1258d3d-7921-44f8-ad81-16c86c8622ba None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.072282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1258d3d-7921-44f8-ad81-16c86c8622ba None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.072925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1258d3d-7921-44f8-ad81-16c86c8622ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.080969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 774/1545] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:16:46.080969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd033e-d16d-4af0-8244-144800d0a687 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.099421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd033e-d16d-4af0-8244-144800d0a687 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.114982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a1258d3d-7921-44f8-ad81-16c86c8622ba None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.116224 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a1258d3d-7921-44f8-ad81-16c86c8622ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.200966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 772/1546] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.203589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb5f2-7459-4848-a655-4e2bb3c9b079 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.208347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb5f2-7459-4848-a655-4e2bb3c9b079 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.208347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb5f2-7459-4848-a655-4e2bb3c9b079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.219991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b484a7b-49bd-4cd5-b452-b88d5154eab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:46.220984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b484a7b-49bd-4cd5-b452-b88d5154eab5 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.221484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b484a7b-49bd-4cd5-b452-b88d5154eab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.221986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b484a7b-49bd-4cd5-b452-b88d5154eab5 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.223904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b484a7b-49bd-4cd5-b452-b88d5154eab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:46.227076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b484a7b-49bd-4cd5-b452-b88d5154eab5 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:46.243945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-523eb5f2-7459-4848-a655-4e2bb3c9b079 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.245832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-523eb5f2-7459-4848-a655-4e2bb3c9b079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.263598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 773/1547] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.263598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523eb5f2-7459-4848-a655-4e2bb3c9b079 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.271657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523eb5f2-7459-4848-a655-4e2bb3c9b079 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.276973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc9ab1-d2c4-4b5d-9d2e-70eec53226fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:46.277953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9ab1-d2c4-4b5d-9d2e-70eec53226fd admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.278244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9ab1-d2c4-4b5d-9d2e-70eec53226fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.278531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9ab1-d2c4-4b5d-9d2e-70eec53226fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.279194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc9ab1-d2c4-4b5d-9d2e-70eec53226fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:16:46.279549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc9ab1-d2c4-4b5d-9d2e-70eec53226fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:46.280579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: WARNING py.warnings [None req-bbbc9ab1-d2c4-4b5d-9d2e-70eec53226fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:16:46.280579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: warnings.warn(msg) Aug 24 10:16:46.280579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: Aug 24 10:16:46.281582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc9ab1-d2c4-4b5d-9d2e-70eec53226fd admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:46.297030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 774/1548] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:16:46.297030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b349f3-387d-460b-834d-c76a97f3e9b4 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.297030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b349f3-387d-460b-834d-c76a97f3e9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.297393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b349f3-387d-460b-834d-c76a97f3e9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.321453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-41b349f3-387d-460b-834d-c76a97f3e9b4 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.322725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-41b349f3-387d-460b-834d-c76a97f3e9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.331148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 775/1549] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.332333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54709bd6-937a-4784-9a26-34c45745ec31 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.332618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54709bd6-937a-4784-9a26-34c45745ec31 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.332923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54709bd6-937a-4784-9a26-34c45745ec31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.352696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-54709bd6-937a-4784-9a26-34c45745ec31 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.353982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-54709bd6-937a-4784-9a26-34c45745ec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.358289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 775/1550] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/users => generated 287 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:46.359721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbaa31-f256-4416-b96a-cb8c8e482c29 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.360021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbaa31-f256-4416-b96a-cb8c8e482c29 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.360298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbaa31-f256-4416-b96a-cb8c8e482c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.367517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 776/1551] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.372254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ab950-9cf7-40fc-a9e6-6eacf22b23c5 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.372254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ab950-9cf7-40fc-a9e6-6eacf22b23c5 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.372254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ab950-9cf7-40fc-a9e6-6eacf22b23c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.387897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b8bbaa31-f256-4416-b96a-cb8c8e482c29 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.389161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b8bbaa31-f256-4416-b96a-cb8c8e482c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.390890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-803ab950-9cf7-40fc-a9e6-6eacf22b23c5 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.392049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-803ab950-9cf7-40fc-a9e6-6eacf22b23c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.397332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 776/1552] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.398086 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbaa31-f256-4416-b96a-cb8c8e482c29 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.406475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 777/1553] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.407626 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479626c0-b82a-40f7-93a5-0498e105d667 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.407917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479626c0-b82a-40f7-93a5-0498e105d667 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.408191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479626c0-b82a-40f7-93a5-0498e105d667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.410920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbaa31-f256-4416-b96a-cb8c8e482c29 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.416011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2755a04-47f4-4ee7-a039-9b24f769b210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:46.416614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755a04-47f4-4ee7-a039-9b24f769b210 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.417643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755a04-47f4-4ee7-a039-9b24f769b210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.417867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755a04-47f4-4ee7-a039-9b24f769b210 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.418515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2755a04-47f4-4ee7-a039-9b24f769b210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:46.419526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: WARNING py.warnings [None req-e2755a04-47f4-4ee7-a039-9b24f769b210 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:16:46.419526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: warnings.warn(msg) Aug 24 10:16:46.419526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: Aug 24 10:16:46.420724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2755a04-47f4-4ee7-a039-9b24f769b210 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:46.428036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-479626c0-b82a-40f7-93a5-0498e105d667 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.429339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-479626c0-b82a-40f7-93a5-0498e105d667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.439034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 778/1554] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.440270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596c600-23e9-4073-a7ed-9988363951bc None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.440563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596c600-23e9-4073-a7ed-9988363951bc None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.440884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596c600-23e9-4073-a7ed-9988363951bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.461140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-0596c600-23e9-4073-a7ed-9988363951bc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.462353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0596c600-23e9-4073-a7ed-9988363951bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.470823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 779/1555] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.471664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596c600-23e9-4073-a7ed-9988363951bc None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.481895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596c600-23e9-4073-a7ed-9988363951bc None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.486578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee108f-9808-4cb1-89f3-93815497375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:46.487163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee108f-9808-4cb1-89f3-93815497375d admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.487444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee108f-9808-4cb1-89f3-93815497375d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.487717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee108f-9808-4cb1-89f3-93815497375d admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.488354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee108f-9808-4cb1-89f3-93815497375d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:46.489979 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee108f-9808-4cb1-89f3-93815497375d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:46.501466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 780/1556] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:46.502309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee108f-9808-4cb1-89f3-93815497375d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.513503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee108f-9808-4cb1-89f3-93815497375d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.521842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b2ee6-6e3d-4ae8-b2d7-0d75ca4bf0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:46.521842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b2ee6-6e3d-4ae8-b2d7-0d75ca4bf0ed admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.521842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b2ee6-6e3d-4ae8-b2d7-0d75ca4bf0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.521842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b2ee6-6e3d-4ae8-b2d7-0d75ca4bf0ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.522274 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b2ee6-6e3d-4ae8-b2d7-0d75ca4bf0ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:16:46.522274 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b2ee6-6e3d-4ae8-b2d7-0d75ca4bf0ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:46.522274 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b2ee6-6e3d-4ae8-b2d7-0d75ca4bf0ed admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:46.534380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 781/1557] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:16:46.535806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e36f4f-935b-4ff9-bcc0-4faee7206b75 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.536090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e36f4f-935b-4ff9-bcc0-4faee7206b75 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.536365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e36f4f-935b-4ff9-bcc0-4faee7206b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.547946 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 777/1558] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/projects => generated 361 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:46.555612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8328bf-0e03-4b1b-9a55-8f22d809c943 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.555612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8328bf-0e03-4b1b-9a55-8f22d809c943 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.555612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8328bf-0e03-4b1b-9a55-8f22d809c943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.556903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-08e36f4f-935b-4ff9-bcc0-4faee7206b75 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.559071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-08e36f4f-935b-4ff9-bcc0-4faee7206b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.570701 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-eb8328bf-0e03-4b1b-9a55-8f22d809c943 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.571877 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-eb8328bf-0e03-4b1b-9a55-8f22d809c943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.601624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:46.619291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 782/1559] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.625231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e36f4f-935b-4ff9-bcc0-4faee7206b75 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.630159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e36f4f-935b-4ff9-bcc0-4faee7206b75 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.634803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eed1a-1116-458c-84fd-fc57ed907018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:46.636623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eed1a-1116-458c-84fd-fc57ed907018 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.636623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eed1a-1116-458c-84fd-fc57ed907018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.636623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eed1a-1116-458c-84fd-fc57ed907018 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.636623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9eed1a-1116-458c-84fd-fc57ed907018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:16:46.637137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9eed1a-1116-458c-84fd-fc57ed907018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:46.638433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9eed1a-1116-458c-84fd-fc57ed907018 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:46.641203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 778/1560] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.642529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71abd0-e8b8-4509-8736-d48872e76d85 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.642801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71abd0-e8b8-4509-8736-d48872e76d85 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.643077 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71abd0-e8b8-4509-8736-d48872e76d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.658737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 783/1561] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:16:46.658737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eed1a-1116-458c-84fd-fc57ed907018 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.662976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eed1a-1116-458c-84fd-fc57ed907018 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.663736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-6e71abd0-e8b8-4509-8736-d48872e76d85 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.664970 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6e71abd0-e8b8-4509-8736-d48872e76d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.668309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce7c07-5a67-4f16-b2fd-39e4bee4ff08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:46.669057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce7c07-5a67-4f16-b2fd-39e4bee4ff08 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.669531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce7c07-5a67-4f16-b2fd-39e4bee4ff08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.669888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce7c07-5a67-4f16-b2fd-39e4bee4ff08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.670642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce7c07-5a67-4f16-b2fd-39e4bee4ff08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:16:46.671020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce7c07-5a67-4f16-b2fd-39e4bee4ff08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:46.672964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce7c07-5a67-4f16-b2fd-39e4bee4ff08 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:46.673191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 779/1562] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.673987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71abd0-e8b8-4509-8736-d48872e76d85 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.683992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71abd0-e8b8-4509-8736-d48872e76d85 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.688774 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0c220-971c-479b-b4c0-c6e7b94bc744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:46.689397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0c220-971c-479b-b4c0-c6e7b94bc744 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.689681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0c220-971c-479b-b4c0-c6e7b94bc744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.689957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0c220-971c-479b-b4c0-c6e7b94bc744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.692720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 784/1563] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:16:46.692720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce7c07-5a67-4f16-b2fd-39e4bee4ff08 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.693001 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0c220-971c-479b-b4c0-c6e7b94bc744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:46.693001 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0c220-971c-479b-b4c0-c6e7b94bc744 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:46.703817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 780/1564] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:46.703998 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce7c07-5a67-4f16-b2fd-39e4bee4ff08 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.704485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0c220-971c-479b-b4c0-c6e7b94bc744 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.709079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7450e-b653-4dd4-88a0-41b74278eb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:46.709821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7450e-b653-4dd4-88a0-41b74278eb44 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.710101 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7450e-b653-4dd4-88a0-41b74278eb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.710821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7450e-b653-4dd4-88a0-41b74278eb44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.710991 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7450e-b653-4dd4-88a0-41b74278eb44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:46.712926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7450e-b653-4dd4-88a0-41b74278eb44 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:46.716985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0c220-971c-479b-b4c0-c6e7b94bc744 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.719374 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171b32c-ccc8-4d64-9ad8-1151f1bfef1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:46.719954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b32c-ccc8-4d64-9ad8-1151f1bfef1e admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.720239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b32c-ccc8-4d64-9ad8-1151f1bfef1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.720511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b32c-ccc8-4d64-9ad8-1151f1bfef1e admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.721155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171b32c-ccc8-4d64-9ad8-1151f1bfef1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:46.723052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171b32c-ccc8-4d64-9ad8-1151f1bfef1e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:46.733534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:46.894195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 785/1565] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/projects => generated 383 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:46.895645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d111-610d-4459-b25d-f51df293a4fb None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.895929 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d111-610d-4459-b25d-f51df293a4fb None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.896206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d111-610d-4459-b25d-f51df293a4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.915635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a0f1d111-610d-4459-b25d-f51df293a4fb None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:46.917005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a0f1d111-610d-4459-b25d-f51df293a4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:46.956242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 781/1566] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/users => generated 281 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:46.957158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171b32c-ccc8-4d64-9ad8-1151f1bfef1e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.967017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171b32c-ccc8-4d64-9ad8-1151f1bfef1e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.977045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 786/1567] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:46.982062 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1d111-610d-4459-b25d-f51df293a4fb None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:46.992005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1d111-610d-4459-b25d-f51df293a4fb None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:46.997552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bb645-9f48-4389-9445-872a4abdd8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:46.998282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bb645-9f48-4389-9445-872a4abdd8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:46.998566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bb645-9f48-4389-9445-872a4abdd8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:46.998839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bb645-9f48-4389-9445-872a4abdd8e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:46.999486 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5bb645-9f48-4389-9445-872a4abdd8e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.001764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5bb645-9f48-4389-9445-872a4abdd8e0 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.026837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4da7-08b4-4931-b115-6a4407859337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.041678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4da7-08b4-4931-b115-6a4407859337 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.041678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4da7-08b4-4931-b115-6a4407859337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.041678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4da7-08b4-4931-b115-6a4407859337 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.041678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca4da7-08b4-4931-b115-6a4407859337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:16:47.041678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca4da7-08b4-4931-b115-6a4407859337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.041678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca4da7-08b4-4931-b115-6a4407859337 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.042826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 782/1568] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:16:47.043753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4da7-08b4-4931-b115-6a4407859337 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.053141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4da7-08b4-4931-b115-6a4407859337 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.058197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bfe21-170b-4930-9b5f-b0ac35483cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.059077 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bfe21-170b-4930-9b5f-b0ac35483cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.059584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bfe21-170b-4930-9b5f-b0ac35483cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.060034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bfe21-170b-4930-9b5f-b0ac35483cd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.061212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bfe21-170b-4930-9b5f-b0ac35483cd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:16:47.061516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bfe21-170b-4930-9b5f-b0ac35483cd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.063091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bfe21-170b-4930-9b5f-b0ac35483cd9 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.075256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 783/1569] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:16:47.076013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bfe21-170b-4930-9b5f-b0ac35483cd9 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.085002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bfe21-170b-4930-9b5f-b0ac35483cd9 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.089763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bd480-6bca-4806-a479-459f3df85a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.090353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bd480-6bca-4806-a479-459f3df85a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.090634 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bd480-6bca-4806-a479-459f3df85a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.090904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bd480-6bca-4806-a479-459f3df85a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.091509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bd480-6bca-4806-a479-459f3df85a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.093407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bd480-6bca-4806-a479-459f3df85a45 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.185956 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 787/1570] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:16:46 2022] POST /identity/v3/projects => generated 367 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:47.186709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bb645-9f48-4389-9445-872a4abdd8e0 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.223645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 784/1571] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:16:47 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:47.223645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bd480-6bca-4806-a479-459f3df85a45 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.249109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bb645-9f48-4389-9445-872a4abdd8e0 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.269751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:47.289583 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bd480-6bca-4806-a479-459f3df85a45 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.294789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964632b-969d-4677-8159-e1f971a12b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.295414 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964632b-969d-4677-8159-e1f971a12b0c admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.295834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964632b-969d-4677-8159-e1f971a12b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.296230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964632b-969d-4677-8159-e1f971a12b0c admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.296901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964632b-969d-4677-8159-e1f971a12b0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.298829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964632b-969d-4677-8159-e1f971a12b0c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.311460 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928b8e9-b8b6-4e9f-8e83-b4b545429300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.312118 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928b8e9-b8b6-4e9f-8e83-b4b545429300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.312395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928b8e9-b8b6-4e9f-8e83-b4b545429300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.312667 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928b8e9-b8b6-4e9f-8e83-b4b545429300 admin admin] PATH_INFO: `/v3/projects/6f98fb63480b4877a4f6f43cbdeccffe/users/9666911cd26444b288abb42f1c0c79e5/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.329665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928b8e9-b8b6-4e9f-8e83-b4b545429300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f98fb63480b4877a4f6f43cbdeccffe, user_id=9666911cd26444b288abb42f1c0c79e5, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.332354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928b8e9-b8b6-4e9f-8e83-b4b545429300 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.462870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 785/1572] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:16:47 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:47.462870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964632b-969d-4677-8159-e1f971a12b0c admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.470300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964632b-969d-4677-8159-e1f971a12b0c admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.475123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce951e3c-623b-48f2-aa32-2af50bfb4182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.475714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce951e3c-623b-48f2-aa32-2af50bfb4182 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.475997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce951e3c-623b-48f2-aa32-2af50bfb4182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.476272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce951e3c-623b-48f2-aa32-2af50bfb4182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.476967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce951e3c-623b-48f2-aa32-2af50bfb4182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.478570 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce951e3c-623b-48f2-aa32-2af50bfb4182 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.489335 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 786/1573] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:47.492902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce951e3c-623b-48f2-aa32-2af50bfb4182 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.496630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 788/1574] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:47 2022] PUT /identity/v3/projects/6f98fb63480b4877a4f6f43cbdeccffe/users/9666911cd26444b288abb42f1c0c79e5/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:47.497327 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928b8e9-b8b6-4e9f-8e83-b4b545429300 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.504202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce951e3c-623b-48f2-aa32-2af50bfb4182 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.533962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:47.561572 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928b8e9-b8b6-4e9f-8e83-b4b545429300 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.561810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9e011-7c82-4380-a19b-4bc904aa7b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.562233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9e011-7c82-4380-a19b-4bc904aa7b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.562517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9e011-7c82-4380-a19b-4bc904aa7b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.562791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9e011-7c82-4380-a19b-4bc904aa7b17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.563466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9e011-7c82-4380-a19b-4bc904aa7b17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:16:47.563837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9e011-7c82-4380-a19b-4bc904aa7b17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.565407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9e011-7c82-4380-a19b-4bc904aa7b17 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.566238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c4766-29c2-434a-9ffa-10ed788467c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.566837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c4766-29c2-434a-9ffa-10ed788467c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.567111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c4766-29c2-434a-9ffa-10ed788467c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.567382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c4766-29c2-434a-9ffa-10ed788467c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.568002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c4766-29c2-434a-9ffa-10ed788467c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.569966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c4766-29c2-434a-9ffa-10ed788467c5 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.577177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 787/1575] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:16:47.578037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9e011-7c82-4380-a19b-4bc904aa7b17 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.587538 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9e011-7c82-4380-a19b-4bc904aa7b17 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.594136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d9e30-883f-4192-b332-0dd52a338d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.594760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d9e30-883f-4192-b332-0dd52a338d54 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.595043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d9e30-883f-4192-b332-0dd52a338d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.595339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d9e30-883f-4192-b332-0dd52a338d54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.595986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d9e30-883f-4192-b332-0dd52a338d54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.597949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d9e30-883f-4192-b332-0dd52a338d54 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.661167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 789/1576] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:16:47 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:47.661993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c4766-29c2-434a-9ffa-10ed788467c5 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.706729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 788/1577] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:16:47 2022] POST /identity/v3/projects => generated 391 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:47.707794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d9e30-883f-4192-b332-0dd52a338d54 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.727077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c4766-29c2-434a-9ffa-10ed788467c5 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.766248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:47.771610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d9e30-883f-4192-b332-0dd52a338d54 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.777082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9ab42-8b09-48cc-863f-4ee14e273a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.777826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9ab42-8b09-48cc-863f-4ee14e273a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.778170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9ab42-8b09-48cc-863f-4ee14e273a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.778371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9ab42-8b09-48cc-863f-4ee14e273a83 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.778986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9ab42-8b09-48cc-863f-4ee14e273a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.780913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9ab42-8b09-48cc-863f-4ee14e273a83 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.784917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca407121-0379-4231-8a1b-8658b0acdea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.785529 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca407121-0379-4231-8a1b-8658b0acdea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.785811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca407121-0379-4231-8a1b-8658b0acdea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.786085 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca407121-0379-4231-8a1b-8658b0acdea7 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.786703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca407121-0379-4231-8a1b-8658b0acdea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.787704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: WARNING py.warnings [None req-ca407121-0379-4231-8a1b-8658b0acdea7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:16:47.787704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: warnings.warn(msg) Aug 24 10:16:47.787704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: Aug 24 10:16:47.788917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca407121-0379-4231-8a1b-8658b0acdea7 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.871654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 789/1578] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:16:47 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:47.872496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9ab42-8b09-48cc-863f-4ee14e273a83 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.882022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9ab42-8b09-48cc-863f-4ee14e273a83 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.887788 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387e29e-7c1d-48b7-ab46-2e5c49d1c7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.888587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387e29e-7c1d-48b7-ab46-2e5c49d1c7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.888934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387e29e-7c1d-48b7-ab46-2e5c49d1c7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.889229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387e29e-7c1d-48b7-ab46-2e5c49d1c7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.889914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0387e29e-7c1d-48b7-ab46-2e5c49d1c7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.891597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0387e29e-7c1d-48b7-ab46-2e5c49d1c7d5 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.894762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 790/1579] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:16:47 2022] POST /identity/v3/users => generated 284 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:47.897787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca407121-0379-4231-8a1b-8658b0acdea7 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.904115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 790/1580] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:47.905425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426efa4d-02e6-4d07-88a1-480e2a5ceea6 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.905719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426efa4d-02e6-4d07-88a1-480e2a5ceea6 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.905997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426efa4d-02e6-4d07-88a1-480e2a5ceea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.909652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca407121-0379-4231-8a1b-8658b0acdea7 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.914385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852bd2f-5cbf-4b37-890b-fbea9d018629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.915101 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852bd2f-5cbf-4b37-890b-fbea9d018629 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.915397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852bd2f-5cbf-4b37-890b-fbea9d018629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.915675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852bd2f-5cbf-4b37-890b-fbea9d018629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.916310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852bd2f-5cbf-4b37-890b-fbea9d018629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.918447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852bd2f-5cbf-4b37-890b-fbea9d018629 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:47.931836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 791/1581] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:47.931836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852bd2f-5cbf-4b37-890b-fbea9d018629 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:47.939528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852bd2f-5cbf-4b37-890b-fbea9d018629 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:47.942624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-426efa4d-02e6-4d07-88a1-480e2a5ceea6 None None] MFA Rules not processed for user `9666911cd26444b288abb42f1c0c79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:47.944185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-426efa4d-02e6-4d07-88a1-480e2a5ceea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:47.944982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e806a7-73df-4212-9485-b3fd0d087c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:47.945779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e806a7-73df-4212-9485-b3fd0d087c04 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:47.946061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e806a7-73df-4212-9485-b3fd0d087c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:47.946344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e806a7-73df-4212-9485-b3fd0d087c04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:47.947008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e806a7-73df-4212-9485-b3fd0d087c04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:47.949561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e806a7-73df-4212-9485-b3fd0d087c04 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.019239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 791/1582] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:48.019991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426efa4d-02e6-4d07-88a1-480e2a5ceea6 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.031678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426efa4d-02e6-4d07-88a1-480e2a5ceea6 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.036268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b4be9-6dab-4462-8199-a50fd54bbc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.036935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b4be9-6dab-4462-8199-a50fd54bbc0d admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.037217 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b4be9-6dab-4462-8199-a50fd54bbc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.037520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b4be9-6dab-4462-8199-a50fd54bbc0d admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.038184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b4be9-6dab-4462-8199-a50fd54bbc0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.038426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 792/1583] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:16:47 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:48.040114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b4be9-6dab-4462-8199-a50fd54bbc0d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.042593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e806a7-73df-4212-9485-b3fd0d087c04 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.050261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:48.105465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e806a7-73df-4212-9485-b3fd0d087c04 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.110298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519770-67d3-4388-82c8-3a8324c0f68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.110893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519770-67d3-4388-82c8-3a8324c0f68a admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.111170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519770-67d3-4388-82c8-3a8324c0f68a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.111440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519770-67d3-4388-82c8-3a8324c0f68a admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.112053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33519770-67d3-4388-82c8-3a8324c0f68a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.114026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33519770-67d3-4388-82c8-3a8324c0f68a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.174373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 792/1584] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:16:48 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:48.175218 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b4be9-6dab-4462-8199-a50fd54bbc0d admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.185407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b4be9-6dab-4462-8199-a50fd54bbc0d admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.190075 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38929455-7def-4d23-822a-5ff611832bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.190656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38929455-7def-4d23-822a-5ff611832bdd admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.190935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38929455-7def-4d23-822a-5ff611832bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.191217 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38929455-7def-4d23-822a-5ff611832bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.191880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38929455-7def-4d23-822a-5ff611832bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.193520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38929455-7def-4d23-822a-5ff611832bdd admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.205913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 793/1585] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:16:48 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:48.205913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519770-67d3-4388-82c8-3a8324c0f68a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.206181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 793/1586] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:48.206181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38929455-7def-4d23-822a-5ff611832bdd admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.216831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38929455-7def-4d23-822a-5ff611832bdd admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.220414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519770-67d3-4388-82c8-3a8324c0f68a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.221605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67cc8f-65e6-476f-8f7f-616c0be4d500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.221756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02d649-6d26-4ec7-b603-89ae7ec219cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.222198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67cc8f-65e6-476f-8f7f-616c0be4d500 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.222351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02d649-6d26-4ec7-b603-89ae7ec219cb admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.222473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67cc8f-65e6-476f-8f7f-616c0be4d500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.222601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02d649-6d26-4ec7-b603-89ae7ec219cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.222727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67cc8f-65e6-476f-8f7f-616c0be4d500 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.222904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02d649-6d26-4ec7-b603-89ae7ec219cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.223356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67cc8f-65e6-476f-8f7f-616c0be4d500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.223615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02d649-6d26-4ec7-b603-89ae7ec219cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.224983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67cc8f-65e6-476f-8f7f-616c0be4d500 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.225297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02d649-6d26-4ec7-b603-89ae7ec219cb admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.236164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 794/1587] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:48.239848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 794/1588] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:48.239965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67cc8f-65e6-476f-8f7f-616c0be4d500 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.240068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02d649-6d26-4ec7-b603-89ae7ec219cb admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.249967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02d649-6d26-4ec7-b603-89ae7ec219cb admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.250120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67cc8f-65e6-476f-8f7f-616c0be4d500 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.254741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08524c9-52d2-4281-93df-ba18e2ff32e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.254892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd475e33-7bc8-4001-88c3-2c39a594ed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.255386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08524c9-52d2-4281-93df-ba18e2ff32e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.255512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd475e33-7bc8-4001-88c3-2c39a594ed90 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.255683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08524c9-52d2-4281-93df-ba18e2ff32e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.255808 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd475e33-7bc8-4001-88c3-2c39a594ed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.255972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08524c9-52d2-4281-93df-ba18e2ff32e6 admin admin] PATH_INFO: `/v3/projects/5b9d28a584bf4858bb183850dcd74963/users/b57c161c80d84d339953bb9256e0f3e9/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.256093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd475e33-7bc8-4001-88c3-2c39a594ed90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.256533 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd475e33-7bc8-4001-88c3-2c39a594ed90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.258470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd475e33-7bc8-4001-88c3-2c39a594ed90 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.275320 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08524c9-52d2-4281-93df-ba18e2ff32e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b9d28a584bf4858bb183850dcd74963, user_id=b57c161c80d84d339953bb9256e0f3e9, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.277184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: WARNING py.warnings [None req-e08524c9-52d2-4281-93df-ba18e2ff32e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:16:48.277184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: warnings.warn(msg) Aug 24 10:16:48.277184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: Aug 24 10:16:48.279148 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08524c9-52d2-4281-93df-ba18e2ff32e6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.350976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 795/1589] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:16:48 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:48.351802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd475e33-7bc8-4001-88c3-2c39a594ed90 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.396951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 795/1590] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:48 2022] PUT /identity/v3/projects/5b9d28a584bf4858bb183850dcd74963/users/b57c161c80d84d339953bb9256e0f3e9/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:48.398097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08524c9-52d2-4281-93df-ba18e2ff32e6 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.414572 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:48.425785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd475e33-7bc8-4001-88c3-2c39a594ed90 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.462044 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08524c9-52d2-4281-93df-ba18e2ff32e6 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.466947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8047fbf-0392-4107-91fa-fb194bcbe4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.467554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8047fbf-0392-4107-91fa-fb194bcbe4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.467837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8047fbf-0392-4107-91fa-fb194bcbe4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.468114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8047fbf-0392-4107-91fa-fb194bcbe4c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.468756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8047fbf-0392-4107-91fa-fb194bcbe4c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.470430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8047fbf-0392-4107-91fa-fb194bcbe4c0 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.481264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 796/1591] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:48.482557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8047fbf-0392-4107-91fa-fb194bcbe4c0 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.484388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe4c24-6dd1-445d-893b-4d1aff380e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.485083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4c24-6dd1-445d-893b-4d1aff380e3b admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.485367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4c24-6dd1-445d-893b-4d1aff380e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.485645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4c24-6dd1-445d-893b-4d1aff380e3b admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.486303 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe4c24-6dd1-445d-893b-4d1aff380e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.488327 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe4c24-6dd1-445d-893b-4d1aff380e3b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.493568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8047fbf-0392-4107-91fa-fb194bcbe4c0 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.499101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6aa7c-ea0a-42c9-8463-f044fdf246b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.499758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6aa7c-ea0a-42c9-8463-f044fdf246b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.500043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6aa7c-ea0a-42c9-8463-f044fdf246b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.505021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6aa7c-ea0a-42c9-8463-f044fdf246b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.505021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6aa7c-ea0a-42c9-8463-f044fdf246b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.505021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6aa7c-ea0a-42c9-8463-f044fdf246b9 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.512701 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 797/1592] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:48.513688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6aa7c-ea0a-42c9-8463-f044fdf246b9 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.523760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6aa7c-ea0a-42c9-8463-f044fdf246b9 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.533662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d54632-8209-4b39-b4e1-77287c1407ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.533662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d54632-8209-4b39-b4e1-77287c1407ad admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.533662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d54632-8209-4b39-b4e1-77287c1407ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.533662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d54632-8209-4b39-b4e1-77287c1407ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.534090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d54632-8209-4b39-b4e1-77287c1407ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.534090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d54632-8209-4b39-b4e1-77287c1407ad admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.544416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 798/1593] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:48.545464 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d54632-8209-4b39-b4e1-77287c1407ad admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.555055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d54632-8209-4b39-b4e1-77287c1407ad admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.560649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c9c73-6dfb-4a92-95c2-aa111c4e2e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.561262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c9c73-6dfb-4a92-95c2-aa111c4e2e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.561541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c9c73-6dfb-4a92-95c2-aa111c4e2e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.561819 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c9c73-6dfb-4a92-95c2-aa111c4e2e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.562433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c9c73-6dfb-4a92-95c2-aa111c4e2e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.564023 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c9c73-6dfb-4a92-95c2-aa111c4e2e00 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.574460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 799/1594] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:48.575368 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c9c73-6dfb-4a92-95c2-aa111c4e2e00 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.584424 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c9c73-6dfb-4a92-95c2-aa111c4e2e00 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.589123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1fb31-7243-48b1-9275-08a532b46ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.590097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1fb31-7243-48b1-9275-08a532b46ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.590377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1fb31-7243-48b1-9275-08a532b46ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.590651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1fb31-7243-48b1-9275-08a532b46ef3 admin admin] PATH_INFO: `/v3/projects/8824664cdd104c308141264985a547a1/users/c94c65ae14e94796a3b3bd0bb29a91b9/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.598781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 796/1595] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:16:48 2022] POST /identity/v3/users => generated 284 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:48.598781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe4c24-6dd1-445d-893b-4d1aff380e3b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.606249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1fb31-7243-48b1-9275-08a532b46ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8824664cdd104c308141264985a547a1, user_id=c94c65ae14e94796a3b3bd0bb29a91b9, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.608933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1fb31-7243-48b1-9275-08a532b46ef3 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.609425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe4c24-6dd1-445d-893b-4d1aff380e3b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.614065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523d7a7-5ef5-4625-a5f0-181ea24a3493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.614654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523d7a7-5ef5-4625-a5f0-181ea24a3493 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.614930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523d7a7-5ef5-4625-a5f0-181ea24a3493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.615199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523d7a7-5ef5-4625-a5f0-181ea24a3493 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.615841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523d7a7-5ef5-4625-a5f0-181ea24a3493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.617782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523d7a7-5ef5-4625-a5f0-181ea24a3493 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.691768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 800/1596] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:48 2022] PUT /identity/v3/projects/8824664cdd104c308141264985a547a1/users/c94c65ae14e94796a3b3bd0bb29a91b9/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:48.693075 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86654f0e-f4bd-474d-b510-e886d2751361 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.693364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86654f0e-f4bd-474d-b510-e886d2751361 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.693642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86654f0e-f4bd-474d-b510-e886d2751361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.721218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 797/1597] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:16:48 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:48.721427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-86654f0e-f4bd-474d-b510-e886d2751361 None None] MFA Rules not processed for user `b57c161c80d84d339953bb9256e0f3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:48.723063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523d7a7-5ef5-4625-a5f0-181ea24a3493 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.723177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-86654f0e-f4bd-474d-b510-e886d2751361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:48.732458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523d7a7-5ef5-4625-a5f0-181ea24a3493 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.757837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:48.788368 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 801/1598] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:48.789171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86654f0e-f4bd-474d-b510-e886d2751361 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.790476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c327be-f3f9-4871-8b08-bcfcb2057144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.791092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c327be-f3f9-4871-8b08-bcfcb2057144 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.791376 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c327be-f3f9-4871-8b08-bcfcb2057144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.792500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c327be-f3f9-4871-8b08-bcfcb2057144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.792837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c327be-f3f9-4871-8b08-bcfcb2057144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.794623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c327be-f3f9-4871-8b08-bcfcb2057144 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.800775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86654f0e-f4bd-474d-b510-e886d2751361 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.808449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 798/1599] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:48.811060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156371c0-1a25-4237-b66f-5103282e3dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.811187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c327be-f3f9-4871-8b08-bcfcb2057144 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.811310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156371c0-1a25-4237-b66f-5103282e3dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.811310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156371c0-1a25-4237-b66f-5103282e3dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.811310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156371c0-1a25-4237-b66f-5103282e3dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.811310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156371c0-1a25-4237-b66f-5103282e3dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.811310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156371c0-1a25-4237-b66f-5103282e3dd1 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.816727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c327be-f3f9-4871-8b08-bcfcb2057144 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.823104 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 802/1600] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:48.823286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db112632-a66d-4fd7-b108-f0a9f37df538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.823286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112632-a66d-4fd7-b108-f0a9f37df538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.823606 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156371c0-1a25-4237-b66f-5103282e3dd1 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.823778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112632-a66d-4fd7-b108-f0a9f37df538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.823778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112632-a66d-4fd7-b108-f0a9f37df538 admin admin] PATH_INFO: `/v3/projects/297fa582000543acbf078376d7010abb/users/71c5ac35123e425e9753f372f9c6bac0/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.834537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156371c0-1a25-4237-b66f-5103282e3dd1 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:48.839737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae6152-ef0e-4d34-8550-c18c1851e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:48.840498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae6152-ef0e-4d34-8550-c18c1851e0aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.840805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae6152-ef0e-4d34-8550-c18c1851e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.841107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae6152-ef0e-4d34-8550-c18c1851e0aa admin admin] PATH_INFO: `/v3/projects/1f6e168f1f704ea79da9d9398e01cfac/users/c63498977927450493f25b373da0cec9/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.841330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db112632-a66d-4fd7-b108-f0a9f37df538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=297fa582000543acbf078376d7010abb, user_id=71c5ac35123e425e9753f372f9c6bac0, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.844109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db112632-a66d-4fd7-b108-f0a9f37df538 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.858725 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae6152-ef0e-4d34-8550-c18c1851e0aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f6e168f1f704ea79da9d9398e01cfac, user_id=c63498977927450493f25b373da0cec9, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:48.861442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae6152-ef0e-4d34-8550-c18c1851e0aa admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:48.960321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 799/1601] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:48 2022] PUT /identity/v3/projects/297fa582000543acbf078376d7010abb/users/71c5ac35123e425e9753f372f9c6bac0/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:48.961219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db112632-a66d-4fd7-b108-f0a9f37df538 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:48.969532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 803/1602] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:48 2022] PUT /identity/v3/projects/1f6e168f1f704ea79da9d9398e01cfac/users/c63498977927450493f25b373da0cec9/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:48.970867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e2b2e-6e11-4303-b3ec-50fb988ed07c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:48.971156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e2b2e-6e11-4303-b3ec-50fb988ed07c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:48.971432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e2b2e-6e11-4303-b3ec-50fb988ed07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:48.997352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-cd9e2b2e-6e11-4303-b3ec-50fb988ed07c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:48.998564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cd9e2b2e-6e11-4303-b3ec-50fb988ed07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:49.011549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:49.029240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db112632-a66d-4fd7-b108-f0a9f37df538 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.060915 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 804/1603] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:49.061848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e2b2e-6e11-4303-b3ec-50fb988ed07c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.072609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e2b2e-6e11-4303-b3ec-50fb988ed07c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.077425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984270b3-d9bf-4500-983b-62d8859c7a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.078035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984270b3-d9bf-4500-983b-62d8859c7a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.078323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984270b3-d9bf-4500-983b-62d8859c7a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.078605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984270b3-d9bf-4500-983b-62d8859c7a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.079249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984270b3-d9bf-4500-983b-62d8859c7a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.081242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984270b3-d9bf-4500-983b-62d8859c7a3a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.086868 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07243645-0fe3-4e4d-a7e3-ab31de632139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.087474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07243645-0fe3-4e4d-a7e3-ab31de632139 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.087752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07243645-0fe3-4e4d-a7e3-ab31de632139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.088073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07243645-0fe3-4e4d-a7e3-ab31de632139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.089427 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07243645-0fe3-4e4d-a7e3-ab31de632139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.091048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07243645-0fe3-4e4d-a7e3-ab31de632139 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.091768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 805/1604] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:49.092692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984270b3-d9bf-4500-983b-62d8859c7a3a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.103196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984270b3-d9bf-4500-983b-62d8859c7a3a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.107578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 800/1605] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:49.107578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07243645-0fe3-4e4d-a7e3-ab31de632139 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.109372 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73932ec4-6c30-4ae7-9461-93e8f431d9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.109985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73932ec4-6c30-4ae7-9461-93e8f431d9fc admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.110268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73932ec4-6c30-4ae7-9461-93e8f431d9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.110542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73932ec4-6c30-4ae7-9461-93e8f431d9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.111405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73932ec4-6c30-4ae7-9461-93e8f431d9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.114078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73932ec4-6c30-4ae7-9461-93e8f431d9fc admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.115890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07243645-0fe3-4e4d-a7e3-ab31de632139 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.120980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060b461-322c-446b-b958-545282505f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.121641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060b461-322c-446b-b958-545282505f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.121924 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060b461-322c-446b-b958-545282505f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.122204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060b461-322c-446b-b958-545282505f4d admin admin] PATH_INFO: `/v3/projects/d4b0c27ad44c455eb42d6a21931d42f8/users/669c94a1591f41ae81d27f608987850f/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.137372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060b461-322c-446b-b958-545282505f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b0c27ad44c455eb42d6a21931d42f8, user_id=669c94a1591f41ae81d27f608987850f, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.140134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060b461-322c-446b-b958-545282505f4d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.274111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 806/1606] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:16:49 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:49.275278 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73932ec4-6c30-4ae7-9461-93e8f431d9fc admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.289528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 801/1607] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:49 2022] PUT /identity/v3/projects/d4b0c27ad44c455eb42d6a21931d42f8/users/669c94a1591f41ae81d27f608987850f/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:49.291287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd54fc-0fb2-410f-a707-a1fa9c8c3970 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.291577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd54fc-0fb2-410f-a707-a1fa9c8c3970 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.291946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd54fc-0fb2-410f-a707-a1fa9c8c3970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.319729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-dedd54fc-0fb2-410f-a707-a1fa9c8c3970 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:49.319729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-dedd54fc-0fb2-410f-a707-a1fa9c8c3970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:49.343076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73932ec4-6c30-4ae7-9461-93e8f431d9fc admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.354840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:49.376983 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 802/1608] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:49.380236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fc8b0-b499-401c-9aa1-bb0947123df0 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.380236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fc8b0-b499-401c-9aa1-bb0947123df0 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.380236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fc8b0-b499-401c-9aa1-bb0947123df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.407719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a168973-d4f4-4c00-b0b0-387b98c78405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.408134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-139fc8b0-b499-401c-9aa1-bb0947123df0 None None] MFA Rules not processed for user `c63498977927450493f25b373da0cec9`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:49.408390 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a168973-d4f4-4c00-b0b0-387b98c78405 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.408673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a168973-d4f4-4c00-b0b0-387b98c78405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.408986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a168973-d4f4-4c00-b0b0-387b98c78405 admin admin] PATH_INFO: `/v3/projects/66a7713fb6804b4db1fbd15fa71afa74/users/388174c87a2245ca89a634f3b177e166/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.409394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-139fc8b0-b499-401c-9aa1-bb0947123df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:49.424432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a168973-d4f4-4c00-b0b0-387b98c78405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a7713fb6804b4db1fbd15fa71afa74, user_id=388174c87a2245ca89a634f3b177e166, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.427175 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a168973-d4f4-4c00-b0b0-387b98c78405 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.464400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:49.474283 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 803/1609] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:49.475209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fc8b0-b499-401c-9aa1-bb0947123df0 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.503361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 807/1610] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:49 2022] PUT /identity/v3/projects/66a7713fb6804b4db1fbd15fa71afa74/users/388174c87a2245ca89a634f3b177e166/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:49.503361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a168973-d4f4-4c00-b0b0-387b98c78405 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.538792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fc8b0-b499-401c-9aa1-bb0947123df0 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.569314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a168973-d4f4-4c00-b0b0-387b98c78405 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.574790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a053e-2d09-4889-bfea-7099d9c35567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.575390 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a053e-2d09-4889-bfea-7099d9c35567 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.575669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a053e-2d09-4889-bfea-7099d9c35567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.575939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a053e-2d09-4889-bfea-7099d9c35567 admin admin] PATH_INFO: `/v3/projects/b2ba59ce19da4557906a458fd4bd412e/users/72dbf0222ec949f6af5ffddd6b407121/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.591793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558e120-0f8b-4985-8be8-30fd60bbefa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.592407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558e120-0f8b-4985-8be8-30fd60bbefa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.592690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558e120-0f8b-4985-8be8-30fd60bbefa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.592806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a053e-2d09-4889-bfea-7099d9c35567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ba59ce19da4557906a458fd4bd412e, user_id=72dbf0222ec949f6af5ffddd6b407121, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.592993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558e120-0f8b-4985-8be8-30fd60bbefa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.595493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a053e-2d09-4889-bfea-7099d9c35567 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.602074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558e120-0f8b-4985-8be8-30fd60bbefa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.603484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558e120-0f8b-4985-8be8-30fd60bbefa9 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.613104 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 804/1611] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:49.613907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558e120-0f8b-4985-8be8-30fd60bbefa9 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.623493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558e120-0f8b-4985-8be8-30fd60bbefa9 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.629636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edcb86-2c79-41ee-ad6f-d3254f53ee23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.630239 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edcb86-2c79-41ee-ad6f-d3254f53ee23 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.630525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edcb86-2c79-41ee-ad6f-d3254f53ee23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.630797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edcb86-2c79-41ee-ad6f-d3254f53ee23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.631433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edcb86-2c79-41ee-ad6f-d3254f53ee23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.633040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edcb86-2c79-41ee-ad6f-d3254f53ee23 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.643982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 805/1612] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:49.645141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edcb86-2c79-41ee-ad6f-d3254f53ee23 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.655504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edcb86-2c79-41ee-ad6f-d3254f53ee23 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.660343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19140c5-951e-44e1-aed5-09917399d3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.660960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19140c5-951e-44e1-aed5-09917399d3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.661249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19140c5-951e-44e1-aed5-09917399d3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.661523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19140c5-951e-44e1-aed5-09917399d3b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.662156 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19140c5-951e-44e1-aed5-09917399d3b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.664057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19140c5-951e-44e1-aed5-09917399d3b9 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.697391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 808/1613] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:49 2022] PUT /identity/v3/projects/b2ba59ce19da4557906a458fd4bd412e/users/72dbf0222ec949f6af5ffddd6b407121/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:49.699038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12050072-d136-4928-a054-aa37c34db496 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.699323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12050072-d136-4928-a054-aa37c34db496 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.699603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12050072-d136-4928-a054-aa37c34db496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.718796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-12050072-d136-4928-a054-aa37c34db496 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:49.720004 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-12050072-d136-4928-a054-aa37c34db496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:49.761765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 806/1614] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:16:49 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:49.762681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19140c5-951e-44e1-aed5-09917399d3b9 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.772777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19140c5-951e-44e1-aed5-09917399d3b9 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.788436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 809/1615] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:49.788436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7150ae-e6f8-42cb-bf34-1f9db32069b7 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.788436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7150ae-e6f8-42cb-bf34-1f9db32069b7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.788436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7150ae-e6f8-42cb-bf34-1f9db32069b7 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.788436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 810/1616] 10.222.0.30 () {54 vars in 962 bytes} [Wed Aug 24 10:16:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:16:49.789622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e297e6b-1692-48dd-b5fd-b096589c7341 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.789700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e297e6b-1692-48dd-b5fd-b096589c7341 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.790145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e297e6b-1692-48dd-b5fd-b096589c7341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.811645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-7e297e6b-1692-48dd-b5fd-b096589c7341 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:49.813622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7e297e6b-1692-48dd-b5fd-b096589c7341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:49.821997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 811/1617] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:49.822763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e297e6b-1692-48dd-b5fd-b096589c7341 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.832496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e297e6b-1692-48dd-b5fd-b096589c7341 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.837532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832ee8e-ca03-44db-9bfa-660b81181fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.838285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832ee8e-ca03-44db-9bfa-660b81181fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.839106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832ee8e-ca03-44db-9bfa-660b81181fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.839436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832ee8e-ca03-44db-9bfa-660b81181fb6 admin admin] PATH_INFO: `/v3/projects/febedfd24c304dbebbfe7d666cfc9f36/users/253456663817463fa73c4fe38f63ad2c/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.843202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f422366-f502-4bdc-bbfc-b9257475e917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.843815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f422366-f502-4bdc-bbfc-b9257475e917 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.844096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f422366-f502-4bdc-bbfc-b9257475e917 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.844367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f422366-f502-4bdc-bbfc-b9257475e917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.851889 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:49.854141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f422366-f502-4bdc-bbfc-b9257475e917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.855366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832ee8e-ca03-44db-9bfa-660b81181fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=febedfd24c304dbebbfe7d666cfc9f36, user_id=253456663817463fa73c4fe38f63ad2c, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.855945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f422366-f502-4bdc-bbfc-b9257475e917 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.858165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832ee8e-ca03-44db-9bfa-660b81181fb6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.865849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 807/1618] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:49.866651 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f422366-f502-4bdc-bbfc-b9257475e917 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.876657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f422366-f502-4bdc-bbfc-b9257475e917 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.881378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d686b-1798-4ea6-b3bd-3365d793876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.884412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d686b-1798-4ea6-b3bd-3365d793876b admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.884412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d686b-1798-4ea6-b3bd-3365d793876b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.884412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d686b-1798-4ea6-b3bd-3365d793876b admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.884412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d686b-1798-4ea6-b3bd-3365d793876b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.884747 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d686b-1798-4ea6-b3bd-3365d793876b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.895728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 808/1619] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:49.896559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d686b-1798-4ea6-b3bd-3365d793876b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.906176 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d686b-1798-4ea6-b3bd-3365d793876b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:49.910845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772ff6a-1394-4032-8a09-e6d70871792b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:49.911433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772ff6a-1394-4032-8a09-e6d70871792b admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.911721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772ff6a-1394-4032-8a09-e6d70871792b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.911999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772ff6a-1394-4032-8a09-e6d70871792b admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.912618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772ff6a-1394-4032-8a09-e6d70871792b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:49.914239 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772ff6a-1394-4032-8a09-e6d70871792b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:49.929227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 809/1620] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:49.929227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ae928-62fc-4c66-8132-9a6dbf295def None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:49.929227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ae928-62fc-4c66-8132-9a6dbf295def None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:49.929227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ae928-62fc-4c66-8132-9a6dbf295def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:49.937052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:49.965109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 812/1621] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:49 2022] PUT /identity/v3/projects/febedfd24c304dbebbfe7d666cfc9f36/users/253456663817463fa73c4fe38f63ad2c/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:49.965817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832ee8e-ca03-44db-9bfa-660b81181fb6 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:49.967524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4832ee8e-ca03-44db-9bfa-660b81181fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:49.969562 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-f33ae928-62fc-4c66-8132-9a6dbf295def None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:49.971097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f33ae928-62fc-4c66-8132-9a6dbf295def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:50.051736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832ee8e-ca03-44db-9bfa-660b81181fb6 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:50.053295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 810/1622] 10.222.0.30 () {58 vars in 1059 bytes} [Wed Aug 24 10:16:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:50.054317 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ae928-62fc-4c66-8132-9a6dbf295def None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:50.056389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c473fc7-0eac-45cd-a31c-91bff71412ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:50.057550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c473fc7-0eac-45cd-a31c-91bff71412ed service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.058075 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c473fc7-0eac-45cd-a31c-91bff71412ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.058664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c473fc7-0eac-45cd-a31c-91bff71412ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.065676 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ae928-62fc-4c66-8132-9a6dbf295def None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:50.069899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1874e2-d87e-40ca-bfc6-4073f607c41b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:50.070518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1874e2-d87e-40ca-bfc6-4073f607c41b service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.070807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1874e2-d87e-40ca-bfc6-4073f607c41b service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.071075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1874e2-d87e-40ca-bfc6-4073f607c41b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.128013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c473fc7-0eac-45cd-a31c-91bff71412ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:50.129545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c473fc7-0eac-45cd-a31c-91bff71412ed service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:50.143301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 813/1623] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:50.143822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1874e2-d87e-40ca-bfc6-4073f607c41b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:50.144265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c473fc7-0eac-45cd-a31c-91bff71412ed service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:50.145594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1874e2-d87e-40ca-bfc6-4073f607c41b service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:50.154215 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c473fc7-0eac-45cd-a31c-91bff71412ed service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:50.157379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 811/1624] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:50.158212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1874e2-d87e-40ca-bfc6-4073f607c41b service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:50.160642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c506e-7293-4e1e-a007-d9fb6e7717a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:50.162933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c506e-7293-4e1e-a007-d9fb6e7717a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.163558 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c506e-7293-4e1e-a007-d9fb6e7717a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.164412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c506e-7293-4e1e-a007-d9fb6e7717a2 admin admin] PATH_INFO: `/v3/projects/b2ba59ce19da4557906a458fd4bd412e/users/72dbf0222ec949f6af5ffddd6b407121/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.166324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c506e-7293-4e1e-a007-d9fb6e7717a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ba59ce19da4557906a458fd4bd412e, user_id=72dbf0222ec949f6af5ffddd6b407121, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:50.168883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1874e2-d87e-40ca-bfc6-4073f607c41b service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:50.170494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c506e-7293-4e1e-a007-d9fb6e7717a2 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:50.173977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b9f19-551c-4239-9363-cbc6b869262e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:50.175034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b9f19-551c-4239-9363-cbc6b869262e admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.175543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b9f19-551c-4239-9363-cbc6b869262e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.176109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b9f19-551c-4239-9363-cbc6b869262e admin admin] PATH_INFO: `/v3/projects/c546811934cf45a4bd9d1f0596754b91/users/4f90608ca85641c298d5d91fd42af79b/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.184333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 814/1625] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:50 2022] PUT /identity/v3/projects/b2ba59ce19da4557906a458fd4bd412e/users/72dbf0222ec949f6af5ffddd6b407121/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:50.185657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d839a-eefd-49f0-b403-a49801866252 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.185942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d839a-eefd-49f0-b403-a49801866252 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.186352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d839a-eefd-49f0-b403-a49801866252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.200592 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b9f19-551c-4239-9363-cbc6b869262e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c546811934cf45a4bd9d1f0596754b91, user_id=4f90608ca85641c298d5d91fd42af79b, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:50.203441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b9f19-551c-4239-9363-cbc6b869262e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:50.224841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ba5d839a-eefd-49f0-b403-a49801866252 None None] MFA Rules not processed for user `253456663817463fa73c4fe38f63ad2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:50.224841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ba5d839a-eefd-49f0-b403-a49801866252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:50.296962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 815/1626] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:50.296962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d839a-eefd-49f0-b403-a49801866252 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:50.304242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 812/1627] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:50 2022] PUT /identity/v3/projects/c546811934cf45a4bd9d1f0596754b91/users/4f90608ca85641c298d5d91fd42af79b/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:50.304242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2d50b-82aa-4cce-98da-703dc0ff5d55 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.304242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2d50b-82aa-4cce-98da-703dc0ff5d55 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.304242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2d50b-82aa-4cce-98da-703dc0ff5d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.318441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:50.352720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-9cb2d50b-82aa-4cce-98da-703dc0ff5d55 None None] MFA Rules not processed for user `72dbf0222ec949f6af5ffddd6b407121`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:50.355567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9cb2d50b-82aa-4cce-98da-703dc0ff5d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:50.413415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d839a-eefd-49f0-b403-a49801866252 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:50.450599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:50.489333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 813/1628] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:50.490223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2d50b-82aa-4cce-98da-703dc0ff5d55 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:50.502198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b38e58-f00b-4a39-a32e-37e5d3367600 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:50.506904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b38e58-f00b-4a39-a32e-37e5d3367600 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.506904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b38e58-f00b-4a39-a32e-37e5d3367600 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.506904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b38e58-f00b-4a39-a32e-37e5d3367600 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.577935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2d50b-82aa-4cce-98da-703dc0ff5d55 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:50.583740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db7c39-a3ee-47f6-94bb-71e2e10b37c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:50.584949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b38e58-f00b-4a39-a32e-37e5d3367600 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:50.585093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db7c39-a3ee-47f6-94bb-71e2e10b37c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.585093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db7c39-a3ee-47f6-94bb-71e2e10b37c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.585093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db7c39-a3ee-47f6-94bb-71e2e10b37c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.585552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db7c39-a3ee-47f6-94bb-71e2e10b37c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:50.586337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b38e58-f00b-4a39-a32e-37e5d3367600 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:50.587606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db7c39-a3ee-47f6-94bb-71e2e10b37c3 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:50.605260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 816/1629] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:50.605260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b38e58-f00b-4a39-a32e-37e5d3367600 service nova] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:50.616841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b38e58-f00b-4a39-a32e-37e5d3367600 service nova] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:50.622030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a145e-8811-4289-94bf-21d1ba216fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:50.622644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a145e-8811-4289-94bf-21d1ba216fff admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.622932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a145e-8811-4289-94bf-21d1ba216fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.623208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a145e-8811-4289-94bf-21d1ba216fff admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.623851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a145e-8811-4289-94bf-21d1ba216fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:50.625594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a145e-8811-4289-94bf-21d1ba216fff admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:50.645726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 817/1630] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:50.645726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a145e-8811-4289-94bf-21d1ba216fff admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:50.743851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:50.757575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 814/1631] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:16:50 2022] POST /identity/v3/projects => generated 369 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:50.758589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db7c39-a3ee-47f6-94bb-71e2e10b37c3 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:50.766292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a145e-8811-4289-94bf-21d1ba216fff admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:50.873737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e889b0-5656-4480-8000-257244374e33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:50.874516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e889b0-5656-4480-8000-257244374e33 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.874870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e889b0-5656-4480-8000-257244374e33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.875594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e889b0-5656-4480-8000-257244374e33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.896255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db7c39-a3ee-47f6-94bb-71e2e10b37c3 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:50.896255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b8ecc-0b74-463e-9358-e75da9aaa168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:50.897001 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b8ecc-0b74-463e-9358-e75da9aaa168 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.897403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b8ecc-0b74-463e-9358-e75da9aaa168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.897789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b8ecc-0b74-463e-9358-e75da9aaa168 admin admin] PATH_INFO: `/v3/projects/c546811934cf45a4bd9d1f0596754b91/users/4f90608ca85641c298d5d91fd42af79b/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.900037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b8ecc-0b74-463e-9358-e75da9aaa168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c546811934cf45a4bd9d1f0596754b91, user_id=4f90608ca85641c298d5d91fd42af79b, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:50.903227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b8ecc-0b74-463e-9358-e75da9aaa168 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:50.924916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 815/1632] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:50 2022] PUT /identity/v3/projects/c546811934cf45a4bd9d1f0596754b91/users/4f90608ca85641c298d5d91fd42af79b/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:50.924916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b8ecc-0b74-463e-9358-e75da9aaa168 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:50.950132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e889b0-5656-4480-8000-257244374e33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:50.952090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e889b0-5656-4480-8000-257244374e33 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:50.964902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 818/1633] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:50.966294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440a46b-b8ee-4f89-babe-991e9f2b7853 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.966597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440a46b-b8ee-4f89-babe-991e9f2b7853 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.966999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440a46b-b8ee-4f89-babe-991e9f2b7853 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.968351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 819/1634] 10.222.0.30 () {54 vars in 912 bytes} [Wed Aug 24 10:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:16:50.969227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdc5e2-9469-41ad-af68-0cd1b75f5381 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:50.969516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdc5e2-9469-41ad-af68-0cd1b75f5381 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:50.969792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdc5e2-9469-41ad-af68-0cd1b75f5381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:50.999557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:51.001899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-07cdc5e2-9469-41ad-af68-0cd1b75f5381 None None] MFA Rules not processed for user `4f90608ca85641c298d5d91fd42af79b`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:51.003167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-07cdc5e2-9469-41ad-af68-0cd1b75f5381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:51.004151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b8ecc-0b74-463e-9358-e75da9aaa168 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:51.009047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe91e84-0923-45e9-a99d-bc500d627d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:51.009749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe91e84-0923-45e9-a99d-bc500d627d7d admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:51.010146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe91e84-0923-45e9-a99d-bc500d627d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:51.011075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe91e84-0923-45e9-a99d-bc500d627d7d admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:51.012020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe91e84-0923-45e9-a99d-bc500d627d7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:51.014378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe91e84-0923-45e9-a99d-bc500d627d7d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:51.083597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 820/1635] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:51.084996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93680d-a629-49f1-83d4-ffade236b47a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:51.085290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93680d-a629-49f1-83d4-ffade236b47a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:51.085705 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93680d-a629-49f1-83d4-ffade236b47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:51.108444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ea93680d-a629-49f1-83d4-ffade236b47a None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:51.108827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ea93680d-a629-49f1-83d4-ffade236b47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:51.148883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:51.194164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 816/1636] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:16:50 2022] POST /identity/v3/users => generated 291 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:16:51.194164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe91e84-0923-45e9-a99d-bc500d627d7d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:51.224173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 821/1637] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:51.225806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93680d-a629-49f1-83d4-ffade236b47a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:51.240748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93680d-a629-49f1-83d4-ffade236b47a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:51.251173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3e32e-ee0d-49fd-b2e8-2e2159d732e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:51.251798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3e32e-ee0d-49fd-b2e8-2e2159d732e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:51.252135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3e32e-ee0d-49fd-b2e8-2e2159d732e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:51.253964 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3e32e-ee0d-49fd-b2e8-2e2159d732e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:51.254971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3e32e-ee0d-49fd-b2e8-2e2159d732e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:51.257459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3e32e-ee0d-49fd-b2e8-2e2159d732e0 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:51.274438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 822/1638] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:51.275640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3e32e-ee0d-49fd-b2e8-2e2159d732e0 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:51.295140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3e32e-ee0d-49fd-b2e8-2e2159d732e0 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:51.303490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb678f96-cd9b-49cb-a3e0-97b494bfc8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:51.304520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb678f96-cd9b-49cb-a3e0-97b494bfc8cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:51.304992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb678f96-cd9b-49cb-a3e0-97b494bfc8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:51.305627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb678f96-cd9b-49cb-a3e0-97b494bfc8cf admin admin] PATH_INFO: `/v3/projects/717a423af77843ba98ae813b6d469de3/users/64f65d1b9dd143e2b69cc32841ac255e/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:51.325098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe91e84-0923-45e9-a99d-bc500d627d7d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:51.325710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb678f96-cd9b-49cb-a3e0-97b494bfc8cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=717a423af77843ba98ae813b6d469de3, user_id=64f65d1b9dd143e2b69cc32841ac255e, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:51.328993 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb678f96-cd9b-49cb-a3e0-97b494bfc8cf admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:51.329135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a059f-d9ca-4e19-aa4e-0f4517568767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:51.329720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a059f-d9ca-4e19-aa4e-0f4517568767 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:51.329999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a059f-d9ca-4e19-aa4e-0f4517568767 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:51.330320 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a059f-d9ca-4e19-aa4e-0f4517568767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:51.341443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a059f-d9ca-4e19-aa4e-0f4517568767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:51.343288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a059f-d9ca-4e19-aa4e-0f4517568767 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:51.368307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 817/1639] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:51 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:51.447464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a059f-d9ca-4e19-aa4e-0f4517568767 service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:51.458129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a059f-d9ca-4e19-aa4e-0f4517568767 service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:51.465401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5aadf-777f-4ad8-b376-04dc63c4ac1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:51.466117 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5aadf-777f-4ad8-b376-04dc63c4ac1f service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:51.466404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5aadf-777f-4ad8-b376-04dc63c4ac1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:51.466680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5aadf-777f-4ad8-b376-04dc63c4ac1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:51.481859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 823/1640] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:51 2022] PUT /identity/v3/projects/717a423af77843ba98ae813b6d469de3/users/64f65d1b9dd143e2b69cc32841ac255e/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:51.485676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb678f96-cd9b-49cb-a3e0-97b494bfc8cf admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:51.487139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5aadf-777f-4ad8-b376-04dc63c4ac1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:51.488604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5aadf-777f-4ad8-b376-04dc63c4ac1f service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:51.535954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 818/1641] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:16:51.693262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb678f96-cd9b-49cb-a3e0-97b494bfc8cf admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:51.710678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb9e2a-68d8-493c-9b74-4fe2d3198e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:51.710678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb9e2a-68d8-493c-9b74-4fe2d3198e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:51.710678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb9e2a-68d8-493c-9b74-4fe2d3198e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:51.710678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb9e2a-68d8-493c-9b74-4fe2d3198e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:51.711039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb9e2a-68d8-493c-9b74-4fe2d3198e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:51.712815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb9e2a-68d8-493c-9b74-4fe2d3198e73 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:51.766960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 824/1642] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:16:51.779265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5aadf-777f-4ad8-b376-04dc63c4ac1f service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:51.825146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5aadf-777f-4ad8-b376-04dc63c4ac1f service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:51.834400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a26be-688e-4b2b-ae19-a743b8534892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:51.836894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a26be-688e-4b2b-ae19-a743b8534892 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:51.836894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a26be-688e-4b2b-ae19-a743b8534892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:51.836894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a26be-688e-4b2b-ae19-a743b8534892 admin admin] PATH_INFO: `/v3/projects/717a423af77843ba98ae813b6d469de3/users/64f65d1b9dd143e2b69cc32841ac255e/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:51.840152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a26be-688e-4b2b-ae19-a743b8534892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=717a423af77843ba98ae813b6d469de3, user_id=64f65d1b9dd143e2b69cc32841ac255e, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:51.843292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a26be-688e-4b2b-ae19-a743b8534892 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:51.862204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 819/1643] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:16:51 2022] PUT /identity/v3/projects/717a423af77843ba98ae813b6d469de3/users/64f65d1b9dd143e2b69cc32841ac255e/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:16:51.878069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac133521-f84a-422c-8e8a-9abaa9baa0c3 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:51.879113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac133521-f84a-422c-8e8a-9abaa9baa0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:51.879604 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac133521-f84a-422c-8e8a-9abaa9baa0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:51.960301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ac133521-f84a-422c-8e8a-9abaa9baa0c3 None None] MFA Rules not processed for user `64f65d1b9dd143e2b69cc32841ac255e`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:51.961624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ac133521-f84a-422c-8e8a-9abaa9baa0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:52.085195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 825/1644] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:52.327340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a26be-688e-4b2b-ae19-a743b8534892 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:52.397831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a26be-688e-4b2b-ae19-a743b8534892 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:52.402974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41155130-373c-4d36-a3d8-6f9aef471e3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:52.402974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41155130-373c-4d36-a3d8-6f9aef471e3d service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:52.403282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41155130-373c-4d36-a3d8-6f9aef471e3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:52.403423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41155130-373c-4d36-a3d8-6f9aef471e3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:52.415008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41155130-373c-4d36-a3d8-6f9aef471e3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:52.433299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41155130-373c-4d36-a3d8-6f9aef471e3d service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:52.692397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 820/1645] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:53.685135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807bb9d-338f-47a1-a550-af03c486a0c5 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:53.685637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807bb9d-338f-47a1-a550-af03c486a0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:53.685703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807bb9d-338f-47a1-a550-af03c486a0c5 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:53.688206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 826/1646] 10.222.0.30 () {54 vars in 962 bytes} [Wed Aug 24 10:16:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:16:53.700181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c6051-a637-458b-b937-766c97dd9314 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:53.700181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c6051-a637-458b-b937-766c97dd9314 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:53.700181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c6051-a637-458b-b937-766c97dd9314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:53.724526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-7a9c6051-a637-458b-b937-766c97dd9314 None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:53.725813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7a9c6051-a637-458b-b937-766c97dd9314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:53.797332 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:53.810718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 821/1647] 10.222.0.30 () {58 vars in 1059 bytes} [Wed Aug 24 10:16:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:16:53.819971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f3dcb-68d6-4dee-acdb-0729863c9790 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:53.820364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f3dcb-68d6-4dee-acdb-0729863c9790 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:53.820744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f3dcb-68d6-4dee-acdb-0729863c9790 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:53.822456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 827/1648] 10.222.0.30 () {54 vars in 912 bytes} [Wed Aug 24 10:16:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:16:53.830360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37233e45-8318-45d9-b388-804857311e83 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:53.830747 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37233e45-8318-45d9-b388-804857311e83 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:53.831122 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37233e45-8318-45d9-b388-804857311e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:53.863313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-37233e45-8318-45d9-b388-804857311e83 None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:53.865163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-37233e45-8318-45d9-b388-804857311e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:53.877580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 822/1649] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:16:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:53.972226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f3dcb-68d6-4dee-acdb-0729863c9790 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:53.995993 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f3dcb-68d6-4dee-acdb-0729863c9790 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:54.009362 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d901af-a6a6-48f2-aa99-505e81f8ef0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:54.010689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d901af-a6a6-48f2-aa99-505e81f8ef0e service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:54.011086 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d901af-a6a6-48f2-aa99-505e81f8ef0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:54.013193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d901af-a6a6-48f2-aa99-505e81f8ef0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:54.035253 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d901af-a6a6-48f2-aa99-505e81f8ef0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:54.035253 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d901af-a6a6-48f2-aa99-505e81f8ef0e service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:54.037939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:54.048975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 828/1650] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:54.243656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37233e45-8318-45d9-b388-804857311e83 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:54.370408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37233e45-8318-45d9-b388-804857311e83 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:54.379004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdebaa5-f761-456e-a7a2-3283de5cca45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:54.379741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdebaa5-f761-456e-a7a2-3283de5cca45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:54.385817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdebaa5-f761-456e-a7a2-3283de5cca45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:54.387298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdebaa5-f761-456e-a7a2-3283de5cca45 admin admin] PATH_INFO: `/v3/users/9666911cd26444b288abb42f1c0c79e5` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:54.391406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdebaa5-f761-456e-a7a2-3283de5cca45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9666911cd26444b288abb42f1c0c79e5)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:54.393431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdebaa5-f761-456e-a7a2-3283de5cca45 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:54.413114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d901af-a6a6-48f2-aa99-505e81f8ef0e service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:54.433011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d901af-a6a6-48f2-aa99-505e81f8ef0e service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:54.456356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:54.457299 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:54.457580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:54.457853 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] PATH_INFO: `/v3/users/b57c161c80d84d339953bb9256e0f3e9` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:54.460652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b57c161c80d84d339953bb9256e0f3e9)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:54.464152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: WARNING py.warnings [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:16:54.464152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: warnings.warn(msg) Aug 24 10:16:54.464152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: Aug 24 10:16:54.465375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:54.717253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-cbdebaa5-f761-456e-a7a2-3283de5cca45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9666911cd26444b288abb42f1c0c79e5'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:54.719336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-cbdebaa5-f761-456e-a7a2-3283de5cca45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9666911cd26444b288abb42f1c0c79e5'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:54.734608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-cbdebaa5-f761-456e-a7a2-3283de5cca45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9666911cd26444b288abb42f1c0c79e5'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:54.736036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-cbdebaa5-f761-456e-a7a2-3283de5cca45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9666911cd26444b288abb42f1c0c79e5'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:54.804568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b57c161c80d84d339953bb9256e0f3e9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:54.805324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b57c161c80d84d339953bb9256e0f3e9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:54.825419 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b57c161c80d84d339953bb9256e0f3e9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:54.871625 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b57c161c80d84d339953bb9256e0f3e9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:54.896149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-cbdebaa5-f761-456e-a7a2-3283de5cca45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9666911cd26444b288abb42f1c0c79e5'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:54.947731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 823/1651] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:16:54 2022] DELETE /identity/v3/users/9666911cd26444b288abb42f1c0c79e5 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:54.953586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007409b-633f-4b03-b6b1-7056339e1417 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:54.954132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007409b-633f-4b03-b6b1-7056339e1417 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:54.957342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007409b-633f-4b03-b6b1-7056339e1417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:54.990741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c007409b-633f-4b03-b6b1-7056339e1417 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:54.992161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c007409b-633f-4b03-b6b1-7056339e1417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:55.124498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a827e310-5fb2-4503-9d25-e06fbd0ec659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b57c161c80d84d339953bb9256e0f3e9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:55.160814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 829/1652] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:16:54 2022] DELETE /identity/v3/users/b57c161c80d84d339953bb9256e0f3e9 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:16:55.162432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ece5a-32d2-4da3-aafb-da880bde67a8 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:55.162726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ece5a-32d2-4da3-aafb-da880bde67a8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:55.163003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ece5a-32d2-4da3-aafb-da880bde67a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:55.206794 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 824/1653] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:55.276372 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-242ece5a-32d2-4da3-aafb-da880bde67a8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:55.279194 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-242ece5a-32d2-4da3-aafb-da880bde67a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:55.281071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007409b-633f-4b03-b6b1-7056339e1417 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:55.282686 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c007409b-633f-4b03-b6b1-7056339e1417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:55.477370 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007409b-633f-4b03-b6b1-7056339e1417 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:55.613052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 830/1654] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:55.741677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ece5a-32d2-4da3-aafb-da880bde67a8 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:55.750956 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-242ece5a-32d2-4da3-aafb-da880bde67a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:55.756185 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff5507-59e6-4c4f-b13a-a7543eaa36c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:55.757640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff5507-59e6-4c4f-b13a-a7543eaa36c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:55.758101 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff5507-59e6-4c4f-b13a-a7543eaa36c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:55.758782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff5507-59e6-4c4f-b13a-a7543eaa36c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:55.781507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff5507-59e6-4c4f-b13a-a7543eaa36c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:55.787870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff5507-59e6-4c4f-b13a-a7543eaa36c1 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:55.790742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ece5a-32d2-4da3-aafb-da880bde67a8 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:55.812366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a84a54-7a45-47ef-9006-63f54a36bed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:55.813050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a84a54-7a45-47ef-9006-63f54a36bed1 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:55.813417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a84a54-7a45-47ef-9006-63f54a36bed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:55.813612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a84a54-7a45-47ef-9006-63f54a36bed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:55.832164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:55.846635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 825/1655] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:16:55.848398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a84a54-7a45-47ef-9006-63f54a36bed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:55.853463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a84a54-7a45-47ef-9006-63f54a36bed1 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:55.887107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 831/1656] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:57.469790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff5507-59e6-4c4f-b13a-a7543eaa36c1 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:57.471797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0aff5507-59e6-4c4f-b13a-a7543eaa36c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:57.486702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a84a54-7a45-47ef-9006-63f54a36bed1 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:57.488208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-01a84a54-7a45-47ef-9006-63f54a36bed1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:57.500541 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff5507-59e6-4c4f-b13a-a7543eaa36c1 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:57.518119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554621db-a68d-41a0-a61a-d8e2b5bf25b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:57.521755 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554621db-a68d-41a0-a61a-d8e2b5bf25b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:57.522107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a84a54-7a45-47ef-9006-63f54a36bed1 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:57.525665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554621db-a68d-41a0-a61a-d8e2b5bf25b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:57.526032 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554621db-a68d-41a0-a61a-d8e2b5bf25b4 admin admin] PATH_INFO: `/v3/users/c63498977927450493f25b373da0cec9` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:57.530580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554621db-a68d-41a0-a61a-d8e2b5bf25b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c63498977927450493f25b373da0cec9)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:57.534869 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554621db-a68d-41a0-a61a-d8e2b5bf25b4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:57.538637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:57.539409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:57.539757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:57.540097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] PATH_INFO: `/v3/projects/5b9d28a584bf4858bb183850dcd74963` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:57.542136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b9d28a584bf4858bb183850dcd74963)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:57.544313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:57.856104 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-554621db-a68d-41a0-a61a-d8e2b5bf25b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c63498977927450493f25b373da0cec9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:57.858940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-554621db-a68d-41a0-a61a-d8e2b5bf25b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c63498977927450493f25b373da0cec9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:57.877334 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-554621db-a68d-41a0-a61a-d8e2b5bf25b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c63498977927450493f25b373da0cec9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:57.878183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-554621db-a68d-41a0-a61a-d8e2b5bf25b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c63498977927450493f25b373da0cec9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:57.894371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b9d28a584bf4858bb183850dcd74963', 'request_id': 'req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:57.905754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:57.923125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b9d28a584bf4858bb183850dcd74963', 'request_id': 'req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.012604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-554621db-a68d-41a0-a61a-d8e2b5bf25b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c63498977927450493f25b373da0cec9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.062984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 826/1657] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:16:57 2022] DELETE /identity/v3/users/c63498977927450493f25b373da0cec9 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:58.067652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738ec97-8874-40eb-9d50-961db83c787f None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:58.070053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738ec97-8874-40eb-9d50-961db83c787f None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:58.070448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738ec97-8874-40eb-9d50-961db83c787f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:58.120971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b9d28a584bf4858bb183850dcd74963', 'request_id': 'req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.141996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] The token cache is being invalidate because project 5b9d28a584bf4858bb183850dcd74963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:16:58.142536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-5738ec97-8874-40eb-9d50-961db83c787f None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:58.143051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.143994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5738ec97-8874-40eb-9d50-961db83c787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:58.145193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.149997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 832/1658] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:16:57 2022] DELETE /identity/v3/projects/5b9d28a584bf4858bb183850dcd74963 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:16:58.156021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:58.158744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:58.330282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 827/1659] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:58.332672 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738ec97-8874-40eb-9d50-961db83c787f None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:58.342329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca2fd2-7ede-41f0-bf2d-e89506aeb59a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:58.342723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5738ec97-8874-40eb-9d50-961db83c787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:58.357719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:58.358416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:58.358697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:58.358971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] PATH_INFO: `/v3/users/4f90608ca85641c298d5d91fd42af79b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:58.360433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f90608ca85641c298d5d91fd42af79b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:58.363865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:58.370916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738ec97-8874-40eb-9d50-961db83c787f None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:58.381649 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:58.382727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:58.383207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:58.383670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] PATH_INFO: `/v3/projects/6f98fb63480b4877a4f6f43cbdeccffe` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:58.386197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f98fb63480b4877a4f6f43cbdeccffe)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:58.387751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: WARNING py.warnings [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:16:58.387751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: warnings.warn(msg) Aug 24 10:16:58.387751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: Aug 24 10:16:58.389623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:58.457422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:58.653434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f90608ca85641c298d5d91fd42af79b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.657229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f90608ca85641c298d5d91fd42af79b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.684900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f90608ca85641c298d5d91fd42af79b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.685728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f98fb63480b4877a4f6f43cbdeccffe', 'request_id': 'req-79b94a0c-f4bd-4737-ad29-488eeec77233'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.743344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f98fb63480b4877a4f6f43cbdeccffe', 'request_id': 'req-79b94a0c-f4bd-4737-ad29-488eeec77233'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.761965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f90608ca85641c298d5d91fd42af79b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.778285 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f98fb63480b4877a4f6f43cbdeccffe', 'request_id': 'req-79b94a0c-f4bd-4737-ad29-488eeec77233'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.908760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f90608ca85641c298d5d91fd42af79b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.923760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 833/1660] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:16:58 2022] DELETE /identity/v3/users/4f90608ca85641c298d5d91fd42af79b => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:58.923760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:58.927015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:58.995552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] The token cache is being invalidate because project 6f98fb63480b4877a4f6f43cbdeccffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:16:58.996006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:58.997315 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:59.002910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 828/1661] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:16:58 2022] DELETE /identity/v3/projects/6f98fb63480b4877a4f6f43cbdeccffe => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:59.004403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:59.007182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:59.130489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2a3a7-bf4c-4f1d-ac89-bf3d302ff68a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:59.136612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc82734-a367-4ac6-82ae-2255d00ed2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:59.137381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc82734-a367-4ac6-82ae-2255d00ed2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:59.138046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc82734-a367-4ac6-82ae-2255d00ed2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:59.138323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc82734-a367-4ac6-82ae-2255d00ed2b1 admin admin] PATH_INFO: `/v3/users/253456663817463fa73c4fe38f63ad2c` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:59.141065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc82734-a367-4ac6-82ae-2255d00ed2b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253456663817463fa73c4fe38f63ad2c)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:59.142149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc82734-a367-4ac6-82ae-2255d00ed2b1 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:59.152834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b94a0c-f4bd-4737-ad29-488eeec77233 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:59.276414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c05a6-e5a9-4710-8ffa-26c1aa03d59d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:59.276683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c05a6-e5a9-4710-8ffa-26c1aa03d59d service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:59.277001 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c05a6-e5a9-4710-8ffa-26c1aa03d59d service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:59.277347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c05a6-e5a9-4710-8ffa-26c1aa03d59d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:59.289855 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-456c05a6-e5a9-4710-8ffa-26c1aa03d59d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:59.316623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c05a6-e5a9-4710-8ffa-26c1aa03d59d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:59.318155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c05a6-e5a9-4710-8ffa-26c1aa03d59d service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:59.363715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 829/1662] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:59.363715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c05a6-e5a9-4710-8ffa-26c1aa03d59d service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:59.363715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-456c05a6-e5a9-4710-8ffa-26c1aa03d59d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:59.392835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c05a6-e5a9-4710-8ffa-26c1aa03d59d service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:59.405602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:59.407521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:59.407768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:59.408047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] PATH_INFO: `/v3/users/72dbf0222ec949f6af5ffddd6b407121` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:59.412487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72dbf0222ec949f6af5ffddd6b407121)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:59.415970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:59.497442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3bc82734-a367-4ac6-82ae-2255d00ed2b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253456663817463fa73c4fe38f63ad2c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:59.501215 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3bc82734-a367-4ac6-82ae-2255d00ed2b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253456663817463fa73c4fe38f63ad2c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:59.519630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3bc82734-a367-4ac6-82ae-2255d00ed2b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253456663817463fa73c4fe38f63ad2c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:59.549768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3bc82734-a367-4ac6-82ae-2255d00ed2b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253456663817463fa73c4fe38f63ad2c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:59.702418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3bc82734-a367-4ac6-82ae-2255d00ed2b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253456663817463fa73c4fe38f63ad2c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:59.713920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 834/1663] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:16:58 2022] DELETE /identity/v3/users/253456663817463fa73c4fe38f63ad2c => generated 0 bytes in 794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:59.718723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae4cee-1059-4168-b002-2987595ac867 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:59.719292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae4cee-1059-4168-b002-2987595ac867 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:59.719785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae4cee-1059-4168-b002-2987595ac867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:59.756112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72dbf0222ec949f6af5ffddd6b407121'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:59.756977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72dbf0222ec949f6af5ffddd6b407121'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:59.761873 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e0ae4cee-1059-4168-b002-2987595ac867 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:59.765264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e0ae4cee-1059-4168-b002-2987595ac867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:59.774571 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72dbf0222ec949f6af5ffddd6b407121'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:59.777998 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72dbf0222ec949f6af5ffddd6b407121'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:59.850059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 835/1664] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:59.871658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72dbf0222ec949f6af5ffddd6b407121'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:16:59.881607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae4cee-1059-4168-b002-2987595ac867 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:59.883465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e0ae4cee-1059-4168-b002-2987595ac867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:59.921702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 830/1665] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:16:59 2022] DELETE /identity/v3/users/72dbf0222ec949f6af5ffddd6b407121 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:16:59.995832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae4cee-1059-4168-b002-2987595ac867 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:00.100703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfff763-0dc7-4a33-bb01-fb1f0a828b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:00.101629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfff763-0dc7-4a33-bb01-fb1f0a828b65 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:00.102030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfff763-0dc7-4a33-bb01-fb1f0a828b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:00.102357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfff763-0dc7-4a33-bb01-fb1f0a828b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:00.235611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfff763-0dc7-4a33-bb01-fb1f0a828b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:00.237552 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfff763-0dc7-4a33-bb01-fb1f0a828b65 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:00.272696 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 836/1666] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:00.731648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:00.902466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:00.904135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:00.928097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfff763-0dc7-4a33-bb01-fb1f0a828b65 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:00.929585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-bcfff763-0dc7-4a33-bb01-fb1f0a828b65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:00.929776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d9114-b05c-4038-9c53-79cb05c12452 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:00.947188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:00.948773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:00.949250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:00.949630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] PATH_INFO: `/v3/projects/1f6e168f1f704ea79da9d9398e01cfac` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:00.951232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfff763-0dc7-4a33-bb01-fb1f0a828b65 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:00.951906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f6e168f1f704ea79da9d9398e01cfac)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:00.961909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:00.963079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:00.963506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:00.963940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] PATH_INFO: `/v3/projects/c546811934cf45a4bd9d1f0596754b91` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:00.970763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:00.979412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c546811934cf45a4bd9d1f0596754b91)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:00.983439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:01.040725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:01.202812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c546811934cf45a4bd9d1f0596754b91', 'request_id': 'req-bebbe518-9f26-42c1-94a4-833452ea1495'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:01.242351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c546811934cf45a4bd9d1f0596754b91', 'request_id': 'req-bebbe518-9f26-42c1-94a4-833452ea1495'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:01.256815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f6e168f1f704ea79da9d9398e01cfac', 'request_id': 'req-6c53c852-3920-4204-903a-d35ec854b5c5'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:01.277631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f6e168f1f704ea79da9d9398e01cfac', 'request_id': 'req-6c53c852-3920-4204-903a-d35ec854b5c5'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:01.297798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f6e168f1f704ea79da9d9398e01cfac', 'request_id': 'req-6c53c852-3920-4204-903a-d35ec854b5c5'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:01.408565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c546811934cf45a4bd9d1f0596754b91', 'request_id': 'req-bebbe518-9f26-42c1-94a4-833452ea1495'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:01.442778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] The token cache is being invalidate because project c546811934cf45a4bd9d1f0596754b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:01.443159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:01.450059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bebbe518-9f26-42c1-94a4-833452ea1495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:01.452526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 837/1667] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:00 2022] DELETE /identity/v3/projects/c546811934cf45a4bd9d1f0596754b91 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:01.458436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfa56d-24f4-44e3-89d3-8779d63cc6cc None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:01.460189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfa56d-24f4-44e3-89d3-8779d63cc6cc None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:01.460189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfa56d-24f4-44e3-89d3-8779d63cc6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:01.506758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-45bfa56d-24f4-44e3-89d3-8779d63cc6cc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:01.507873 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-45bfa56d-24f4-44e3-89d3-8779d63cc6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:01.557359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] The token cache is being invalidate because project 1f6e168f1f704ea79da9d9398e01cfac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:01.557714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:01.560741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6c53c852-3920-4204-903a-d35ec854b5c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:01.566125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 831/1668] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:00 2022] DELETE /identity/v3/projects/1f6e168f1f704ea79da9d9398e01cfac => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:01.567739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af179f-de55-48b9-8653-3dc371dd3929 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:01.568625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af179f-de55-48b9-8653-3dc371dd3929 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:01.568997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af179f-de55-48b9-8653-3dc371dd3929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:01.625539 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-75af179f-de55-48b9-8653-3dc371dd3929 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:01.630263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-75af179f-de55-48b9-8653-3dc371dd3929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:01.742504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 838/1669] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:01.751032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfa56d-24f4-44e3-89d3-8779d63cc6cc None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:01.757970 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-45bfa56d-24f4-44e3-89d3-8779d63cc6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:01.798491 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfa56d-24f4-44e3-89d3-8779d63cc6cc None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:01.818346 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:01.819549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:01.819549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:01.819549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] PATH_INFO: `/v3/projects/b2ba59ce19da4557906a458fd4bd412e` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:01.834772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2ba59ce19da4557906a458fd4bd412e)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:01.842965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:01.858556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 832/1670] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:01.863940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39669884-7ae7-4a85-b91c-9881458c6535 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:01.864939 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39669884-7ae7-4a85-b91c-9881458c6535 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:01.867078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39669884-7ae7-4a85-b91c-9881458c6535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:01.937499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-39669884-7ae7-4a85-b91c-9881458c6535 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:01.940368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-39669884-7ae7-4a85-b91c-9881458c6535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:01.955991 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 833/1671] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:01.957717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f152f8b-298d-40ce-8d5a-1d3802d24208 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:01.958970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f152f8b-298d-40ce-8d5a-1d3802d24208 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:01.959361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f152f8b-298d-40ce-8d5a-1d3802d24208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:02.003926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-8f152f8b-298d-40ce-8d5a-1d3802d24208 None None] MFA Rules not processed for user `388174c87a2245ca89a634f3b177e166`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:02.005984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8f152f8b-298d-40ce-8d5a-1d3802d24208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:02.075711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2ba59ce19da4557906a458fd4bd412e', 'request_id': 'req-daa57c21-1d6e-48d8-b896-b4fb17190f70'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:02.093871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2ba59ce19da4557906a458fd4bd412e', 'request_id': 'req-daa57c21-1d6e-48d8-b896-b4fb17190f70'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:02.163875 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 834/1672] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:02.170081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09535a0-21cb-42d3-ac3b-8506651f14bf None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:02.170081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09535a0-21cb-42d3-ac3b-8506651f14bf None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:02.170081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09535a0-21cb-42d3-ac3b-8506651f14bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:02.197012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-f09535a0-21cb-42d3-ac3b-8506651f14bf None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:02.198600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f09535a0-21cb-42d3-ac3b-8506651f14bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:02.229405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2ba59ce19da4557906a458fd4bd412e', 'request_id': 'req-daa57c21-1d6e-48d8-b896-b4fb17190f70'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:02.252241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] The token cache is being invalidate because project b2ba59ce19da4557906a458fd4bd412e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:02.252979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:02.255423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:02.265529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 839/1673] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:01 2022] DELETE /identity/v3/projects/b2ba59ce19da4557906a458fd4bd412e => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:02.267107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:02.286047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:02.370358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 835/1674] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:02.371415 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09535a0-21cb-42d3-ac3b-8506651f14bf None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:02.377501 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f09535a0-21cb-42d3-ac3b-8506651f14bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:02.403591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09535a0-21cb-42d3-ac3b-8506651f14bf None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:02.418271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:02.418938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:02.419226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:02.419499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] PATH_INFO: `/v3/projects/febedfd24c304dbebbfe7d666cfc9f36` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:02.420838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=febedfd24c304dbebbfe7d666cfc9f36)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:02.422972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:02.473765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa57c21-1d6e-48d8-b896-b4fb17190f70 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:02.502702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca0115-3767-413f-9f81-d20188753221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:02.502702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca0115-3767-413f-9f81-d20188753221 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:02.502702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca0115-3767-413f-9f81-d20188753221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:02.502702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca0115-3767-413f-9f81-d20188753221 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:02.503130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca0115-3767-413f-9f81-d20188753221 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:02.503130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca0115-3767-413f-9f81-d20188753221 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:02.503130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca0115-3767-413f-9f81-d20188753221 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:02.517708 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 840/1675] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:02.519940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014f1d5-4dc4-4c6a-9ae0-afb0f7c5780b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:02.520851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014f1d5-4dc4-4c6a-9ae0-afb0f7c5780b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:02.521367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014f1d5-4dc4-4c6a-9ae0-afb0f7c5780b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:02.568004 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b014f1d5-4dc4-4c6a-9ae0-afb0f7c5780b None None] MFA Rules not processed for user `388174c87a2245ca89a634f3b177e166`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:02.571057 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b014f1d5-4dc4-4c6a-9ae0-afb0f7c5780b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:02.684549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 841/1676] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:02.685837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d1d60-1e77-4179-aaa7-f9f9afccb992 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:02.686443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d1d60-1e77-4179-aaa7-f9f9afccb992 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:02.686631 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d1d60-1e77-4179-aaa7-f9f9afccb992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:02.735035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3b0d1d60-1e77-4179-aaa7-f9f9afccb992 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:02.737539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3b0d1d60-1e77-4179-aaa7-f9f9afccb992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:02.751028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'febedfd24c304dbebbfe7d666cfc9f36', 'request_id': 'req-299d08fe-28bf-4567-8806-9edb54010734'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:02.782081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'febedfd24c304dbebbfe7d666cfc9f36', 'request_id': 'req-299d08fe-28bf-4567-8806-9edb54010734'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:02.795297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'febedfd24c304dbebbfe7d666cfc9f36', 'request_id': 'req-299d08fe-28bf-4567-8806-9edb54010734'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:02.873170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 842/1677] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:02.874034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d1d60-1e77-4179-aaa7-f9f9afccb992 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:02.894729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d1d60-1e77-4179-aaa7-f9f9afccb992 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:02.905170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b1dfc-178e-4c5c-93d2-aa00c38eb2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:02.905771 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b1dfc-178e-4c5c-93d2-aa00c38eb2fc admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:02.906053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b1dfc-178e-4c5c-93d2-aa00c38eb2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:02.906329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b1dfc-178e-4c5c-93d2-aa00c38eb2fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:02.907004 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b1dfc-178e-4c5c-93d2-aa00c38eb2fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:02.907840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b1dfc-178e-4c5c-93d2-aa00c38eb2fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:02.909007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b1dfc-178e-4c5c-93d2-aa00c38eb2fc admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:02.944737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 843/1678] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:02.944737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b1dfc-178e-4c5c-93d2-aa00c38eb2fc admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:02.947440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] The token cache is being invalidate because project febedfd24c304dbebbfe7d666cfc9f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:02.947965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:02.949450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:02.952354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 836/1679] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:02 2022] DELETE /identity/v3/projects/febedfd24c304dbebbfe7d666cfc9f36 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:02.953697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:02.956811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:02.984165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b1dfc-178e-4c5c-93d2-aa00c38eb2fc admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:02.996681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d46e0-3828-45ab-a82d-8a488e1cc0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:02.999384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d46e0-3828-45ab-a82d-8a488e1cc0cc admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:02.999690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d46e0-3828-45ab-a82d-8a488e1cc0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:03.000991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d46e0-3828-45ab-a82d-8a488e1cc0cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:03.001645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d46e0-3828-45ab-a82d-8a488e1cc0cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:03.005367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d46e0-3828-45ab-a82d-8a488e1cc0cc admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:03.115624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d08fe-28bf-4567-8806-9edb54010734 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:03.124387 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670e17-61df-4226-b31f-702f4871f981 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:03.125379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670e17-61df-4226-b31f-702f4871f981 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:03.125859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670e17-61df-4226-b31f-702f4871f981 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:03.126496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670e17-61df-4226-b31f-702f4871f981 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:03.129558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-dc670e17-61df-4226-b31f-702f4871f981 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:03.195961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 844/1680] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:02 2022] POST /identity/v3/projects => generated 387 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:03.197886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db980e8e-cd53-4a98-b7c6-8070bff89f90 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:03.199743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db980e8e-cd53-4a98-b7c6-8070bff89f90 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:03.200220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db980e8e-cd53-4a98-b7c6-8070bff89f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:03.219575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc670e17-61df-4226-b31f-702f4871f981 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:03.221666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc670e17-61df-4226-b31f-702f4871f981 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:03.225999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:03.241855 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 837/1681] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:03.246255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670e17-61df-4226-b31f-702f4871f981 service nova] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:03.249197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-db980e8e-cd53-4a98-b7c6-8070bff89f90 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:03.250114 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-dc670e17-61df-4226-b31f-702f4871f981 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:03.253197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-db980e8e-cd53-4a98-b7c6-8070bff89f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:03.283145 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670e17-61df-4226-b31f-702f4871f981 service nova] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:03.387966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2dacea-942f-4f1b-ad0a-5bfe8ad3181a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:03.387966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dacea-942f-4f1b-ad0a-5bfe8ad3181a admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:03.387966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dacea-942f-4f1b-ad0a-5bfe8ad3181a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:03.388294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 845/1682] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:03.388294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db980e8e-cd53-4a98-b7c6-8070bff89f90 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:03.388888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dacea-942f-4f1b-ad0a-5bfe8ad3181a admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:03.389603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2dacea-942f-4f1b-ad0a-5bfe8ad3181a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:03.391523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2dacea-942f-4f1b-ad0a-5bfe8ad3181a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:03.394356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-db980e8e-cd53-4a98-b7c6-8070bff89f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:03.440448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db980e8e-cd53-4a98-b7c6-8070bff89f90 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:03.460219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:03.461123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:03.462572 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:03.462572 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] PATH_INFO: `/v3/users/64f65d1b9dd143e2b69cc32841ac255e` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:03.464702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:03.467109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] RBAC: Authorizing `identity:delete_user(user_id=64f65d1b9dd143e2b69cc32841ac255e)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:03.469306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:03.531211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 838/1683] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:03 2022] POST /identity/v3/projects => generated 379 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:03.539907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2dacea-942f-4f1b-ad0a-5bfe8ad3181a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:03.543031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7e2dacea-942f-4f1b-ad0a-5bfe8ad3181a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:03.704213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2dacea-942f-4f1b-ad0a-5bfe8ad3181a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:03.721996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358e858-d7d0-422e-9348-470a487faa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:03.722652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358e858-d7d0-422e-9348-470a487faa77 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:03.722994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358e858-d7d0-422e-9348-470a487faa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:03.723275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358e858-d7d0-422e-9348-470a487faa77 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:03.724898 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358e858-d7d0-422e-9348-470a487faa77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:03.732570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358e858-d7d0-422e-9348-470a487faa77 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:03.772729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64f65d1b9dd143e2b69cc32841ac255e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:03.773495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64f65d1b9dd143e2b69cc32841ac255e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:03.797477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64f65d1b9dd143e2b69cc32841ac255e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:03.857646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64f65d1b9dd143e2b69cc32841ac255e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:03.872468 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 839/1684] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:03 2022] POST /identity/v3/users => generated 294 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:03.873829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358e858-d7d0-422e-9348-470a487faa77 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:03.912210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358e858-d7d0-422e-9348-470a487faa77 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:03.928787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3d05d-544f-4cb0-a7ae-3d8d2fb44d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:03.933716 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3d05d-544f-4cb0-a7ae-3d8d2fb44d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:03.934117 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3d05d-544f-4cb0-a7ae-3d8d2fb44d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:03.935554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3d05d-544f-4cb0-a7ae-3d8d2fb44d75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:03.936408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3d05d-544f-4cb0-a7ae-3d8d2fb44d75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:03.942443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3d05d-544f-4cb0-a7ae-3d8d2fb44d75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:03.944583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3d05d-544f-4cb0-a7ae-3d8d2fb44d75 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:04.000688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 840/1685] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:04.007047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3d05d-544f-4cb0-a7ae-3d8d2fb44d75 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:04.028069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64f65d1b9dd143e2b69cc32841ac255e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:04.034072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3d05d-544f-4cb0-a7ae-3d8d2fb44d75 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:04.039665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 846/1686] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:03 2022] DELETE /identity/v3/users/64f65d1b9dd143e2b69cc32841ac255e => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:04.040720 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:04.049344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:04.050878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349ee84-a05f-49ec-9fce-2d140e29822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:04.051723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349ee84-a05f-49ec-9fce-2d140e29822c admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.052119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349ee84-a05f-49ec-9fce-2d140e29822c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.052499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349ee84-a05f-49ec-9fce-2d140e29822c admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:04.053978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349ee84-a05f-49ec-9fce-2d140e29822c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:04.059773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349ee84-a05f-49ec-9fce-2d140e29822c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:04.229801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 841/1687] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:04 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:04.231120 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349ee84-a05f-49ec-9fce-2d140e29822c admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:04.238838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-4349ee84-a05f-49ec-9fce-2d140e29822c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:04.258868 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa014e6c-68c5-4204-bc64-90b7aacb501d admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:04.292303 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349ee84-a05f-49ec-9fce-2d140e29822c admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:04.306344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc1902-80a4-454c-ac34-6f86e95e3c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:04.307960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc1902-80a4-454c-ac34-6f86e95e3c7e admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.308298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc1902-80a4-454c-ac34-6f86e95e3c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.308587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc1902-80a4-454c-ac34-6f86e95e3c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:04.309392 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc1902-80a4-454c-ac34-6f86e95e3c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:04.314965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02667b23-af95-4a23-85ad-fd5170735eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:04.317828 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02667b23-af95-4a23-85ad-fd5170735eba admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.318269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02667b23-af95-4a23-85ad-fd5170735eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.318810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02667b23-af95-4a23-85ad-fd5170735eba admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:04.319646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02667b23-af95-4a23-85ad-fd5170735eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:04.322049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc1902-80a4-454c-ac34-6f86e95e3c7e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:04.326699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02667b23-af95-4a23-85ad-fd5170735eba admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:04.376433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 847/1688] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:04.376433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02667b23-af95-4a23-85ad-fd5170735eba admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:04.381434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-02667b23-af95-4a23-85ad-fd5170735eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:04.423830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02667b23-af95-4a23-85ad-fd5170735eba admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:04.444845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d2343-c4fc-4e08-a26c-0a31c362728b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:04.445669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2343-c4fc-4e08-a26c-0a31c362728b admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.445951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2343-c4fc-4e08-a26c-0a31c362728b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.446230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2343-c4fc-4e08-a26c-0a31c362728b admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:04.446890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d2343-c4fc-4e08-a26c-0a31c362728b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:04.452183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d2343-c4fc-4e08-a26c-0a31c362728b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:04.495316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 848/1689] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:04.495316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d2343-c4fc-4e08-a26c-0a31c362728b admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:04.502123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 842/1690] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:04 2022] POST /identity/v3/projects => generated 365 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:04.503317 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc1902-80a4-454c-ac34-6f86e95e3c7e admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:04.526102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d2343-c4fc-4e08-a26c-0a31c362728b admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:04.685229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad02a9-97fd-4391-8a20-fc592bfe61a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:04.685867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad02a9-97fd-4391-8a20-fc592bfe61a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.686149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad02a9-97fd-4391-8a20-fc592bfe61a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.686428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad02a9-97fd-4391-8a20-fc592bfe61a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:04.687087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad02a9-97fd-4391-8a20-fc592bfe61a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:04.688759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad02a9-97fd-4391-8a20-fc592bfe61a6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:04.690146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc1902-80a4-454c-ac34-6f86e95e3c7e admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:04.711765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a56e9-ccf9-46f7-8fe3-65d137594c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:04.712494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a56e9-ccf9-46f7-8fe3-65d137594c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.713084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a56e9-ccf9-46f7-8fe3-65d137594c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.713084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a56e9-ccf9-46f7-8fe3-65d137594c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:04.713923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a56e9-ccf9-46f7-8fe3-65d137594c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:04.715901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 849/1691] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:04.716765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad02a9-97fd-4391-8a20-fc592bfe61a6 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:04.717222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a56e9-ccf9-46f7-8fe3-65d137594c34 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:04.736830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad02a9-97fd-4391-8a20-fc592bfe61a6 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:04.749059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 843/1692] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:04.749780 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a56e9-ccf9-46f7-8fe3-65d137594c34 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:04.754597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5f483-dc4f-4e25-9e49-6d20d46a5273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:04.755413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5f483-dc4f-4e25-9e49-6d20d46a5273 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.755585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5f483-dc4f-4e25-9e49-6d20d46a5273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.755847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5f483-dc4f-4e25-9e49-6d20d46a5273 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:04.757700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5f483-dc4f-4e25-9e49-6d20d46a5273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:04.760998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5f483-dc4f-4e25-9e49-6d20d46a5273 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:04.792990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a56e9-ccf9-46f7-8fe3-65d137594c34 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:04.821114 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd69ba6-442c-4442-aac8-65b4750c91db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:04.825011 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd69ba6-442c-4442-aac8-65b4750c91db admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.825545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd69ba6-442c-4442-aac8-65b4750c91db admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.826974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd69ba6-442c-4442-aac8-65b4750c91db admin admin] PATH_INFO: `/v3/projects/5cf27cd5d989481fbfe3f0db7f95eefe/users/f6123c5c34a5457bb6c1ae34501bc20c/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:04.875737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd69ba6-442c-4442-aac8-65b4750c91db admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf27cd5d989481fbfe3f0db7f95eefe, user_id=f6123c5c34a5457bb6c1ae34501bc20c, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:04.879198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd69ba6-442c-4442-aac8-65b4750c91db admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:04.934190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 850/1693] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:04 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:04.942155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8308e4e-051f-42de-95dd-d93ba37f10a7 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.942721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8308e4e-051f-42de-95dd-d93ba37f10a7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.944285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8308e4e-051f-42de-95dd-d93ba37f10a7 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:04.952942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 851/1694] 10.222.0.30 () {54 vars in 912 bytes} [Wed Aug 24 10:17:04 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:17:04.954126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8308e4e-051f-42de-95dd-d93ba37f10a7 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:04.980144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8308e4e-051f-42de-95dd-d93ba37f10a7 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:04.990957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d358f7-b09f-4c5b-9e64-f13c0af8fe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:04.991830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d358f7-b09f-4c5b-9e64-f13c0af8fe2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.992215 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d358f7-b09f-4c5b-9e64-f13c0af8fe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.992599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d358f7-b09f-4c5b-9e64-f13c0af8fe2c admin admin] PATH_INFO: `/v3/projects/5309ab7ca500477685fe9e5187b11db2/users/831114b04211479b8a73677d6ed018e9/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.028534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d358f7-b09f-4c5b-9e64-f13c0af8fe2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5309ab7ca500477685fe9e5187b11db2, user_id=831114b04211479b8a73677d6ed018e9, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.036982 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d358f7-b09f-4c5b-9e64-f13c0af8fe2c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.083687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 844/1695] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:04 2022] PUT /identity/v3/projects/5cf27cd5d989481fbfe3f0db7f95eefe/users/f6123c5c34a5457bb6c1ae34501bc20c/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:05.084250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd69ba6-442c-4442-aac8-65b4750c91db admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.226303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 852/1696] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:04 2022] PUT /identity/v3/projects/5309ab7ca500477685fe9e5187b11db2/users/831114b04211479b8a73677d6ed018e9/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:05.226303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8629b2-6566-470d-9f82-08620253c251 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.226303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8629b2-6566-470d-9f82-08620253c251 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.226303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8629b2-6566-470d-9f82-08620253c251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.315456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd69ba6-442c-4442-aac8-65b4750c91db admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:05.376768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0b8629b2-6566-470d-9f82-08620253c251 None None] MFA Rules not processed for user `ae9b6525009b41288dd918437acee92a`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:05.378284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0b8629b2-6566-470d-9f82-08620253c251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:05.430000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674449d-2ad0-4468-9ff7-36c8659f92e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:05.430801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674449d-2ad0-4468-9ff7-36c8659f92e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.431234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674449d-2ad0-4468-9ff7-36c8659f92e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.432297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674449d-2ad0-4468-9ff7-36c8659f92e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.433545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674449d-2ad0-4468-9ff7-36c8659f92e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.435300 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674449d-2ad0-4468-9ff7-36c8659f92e0 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.455282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 845/1697] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:05.456960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d8516-46e3-4a8c-be44-3cb13a048a65 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.457375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d8516-46e3-4a8c-be44-3cb13a048a65 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.457770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d8516-46e3-4a8c-be44-3cb13a048a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.534483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-062d8516-46e3-4a8c-be44-3cb13a048a65 None None] MFA Rules not processed for user `f6123c5c34a5457bb6c1ae34501bc20c`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:05.538105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-062d8516-46e3-4a8c-be44-3cb13a048a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:05.554477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 853/1698] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:05.556104 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c577d7-2f5d-42f2-9ab9-d96f2789345f None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.567740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c577d7-2f5d-42f2-9ab9-d96f2789345f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.568491 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c577d7-2f5d-42f2-9ab9-d96f2789345f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.642584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e7c577d7-2f5d-42f2-9ab9-d96f2789345f None None] MFA Rules not processed for user `c94c65ae14e94796a3b3bd0bb29a91b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:05.643574 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e7c577d7-2f5d-42f2-9ab9-d96f2789345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:05.710458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 846/1699] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:05.712066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910679-66d7-426d-a071-547923bda83a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.712487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910679-66d7-426d-a071-547923bda83a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.712920 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910679-66d7-426d-a071-547923bda83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.779056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-5b910679-66d7-426d-a071-547923bda83a None None] MFA Rules not processed for user `831114b04211479b8a73677d6ed018e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:05.781012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5b910679-66d7-426d-a071-547923bda83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:05.850617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 854/1700] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:05.852410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c577d7-2f5d-42f2-9ab9-d96f2789345f None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.896660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c577d7-2f5d-42f2-9ab9-d96f2789345f None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:05.914690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445e27a-d93e-4f99-afcf-3d04088ece6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:05.915194 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445e27a-d93e-4f99-afcf-3d04088ece6a admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.915512 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445e27a-d93e-4f99-afcf-3d04088ece6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.916000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445e27a-d93e-4f99-afcf-3d04088ece6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.917466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445e27a-d93e-4f99-afcf-3d04088ece6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.920196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445e27a-d93e-4f99-afcf-3d04088ece6a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.923099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 847/1701] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:05.923840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b910679-66d7-426d-a071-547923bda83a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.934932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5b910679-66d7-426d-a071-547923bda83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:05.939108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b910679-66d7-426d-a071-547923bda83a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:05.946389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd743bc-b122-453b-aba9-f01f2ce51210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:05.947667 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd743bc-b122-453b-aba9-f01f2ce51210 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.947993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd743bc-b122-453b-aba9-f01f2ce51210 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.948228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd743bc-b122-453b-aba9-f01f2ce51210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.958416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd743bc-b122-453b-aba9-f01f2ce51210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.960354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd743bc-b122-453b-aba9-f01f2ce51210 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.964368 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 855/1702] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:05.965343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445e27a-d93e-4f99-afcf-3d04088ece6a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.975841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 848/1703] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:05.976852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd743bc-b122-453b-aba9-f01f2ce51210 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.978532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ddd743bc-b122-453b-aba9-f01f2ce51210 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:05.984915 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445e27a-d93e-4f99-afcf-3d04088ece6a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:05.991036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcd65f-a91d-47f2-93c4-f46392cbc5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:05.992390 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd65f-a91d-47f2-93c4-f46392cbc5de admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.993022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd65f-a91d-47f2-93c4-f46392cbc5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.993350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd65f-a91d-47f2-93c4-f46392cbc5de admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.994833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcd65f-a91d-47f2-93c4-f46392cbc5de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.997179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcd65f-a91d-47f2-93c4-f46392cbc5de admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.143170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd743bc-b122-453b-aba9-f01f2ce51210 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.156343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23b999-a2f1-4c8f-a8ce-8a534e2e2091 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.158757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b999-a2f1-4c8f-a8ce-8a534e2e2091 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.159053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b999-a2f1-4c8f-a8ce-8a534e2e2091 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.159335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b999-a2f1-4c8f-a8ce-8a534e2e2091 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.184652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 856/1704] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:05 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:06.186264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcd65f-a91d-47f2-93c4-f46392cbc5de admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.192154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d0dcd65f-a91d-47f2-93c4-f46392cbc5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:06.202348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23b999-a2f1-4c8f-a8ce-8a534e2e2091 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.210416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23b999-a2f1-4c8f-a8ce-8a534e2e2091 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.235339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 849/1705] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:06.237510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:06.238345 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23b999-a2f1-4c8f-a8ce-8a534e2e2091 service nova] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.268339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:06.270317 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23b999-a2f1-4c8f-a8ce-8a534e2e2091 service nova] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.403439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcd65f-a91d-47f2-93c4-f46392cbc5de admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.417448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.418168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.418555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.419524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] PATH_INFO: `/v3/projects/717a423af77843ba98ae813b6d469de3` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.423011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=717a423af77843ba98ae813b6d469de3)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.429018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.430591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2be9e-e766-4d86-b3c0-c42c59cce3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.431009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2be9e-e766-4d86-b3c0-c42c59cce3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.431316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2be9e-e766-4d86-b3c0-c42c59cce3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.431604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2be9e-e766-4d86-b3c0-c42c59cce3bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.433358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2be9e-e766-4d86-b3c0-c42c59cce3bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.441398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2be9e-e766-4d86-b3c0-c42c59cce3bd admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.563533 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '717a423af77843ba98ae813b6d469de3', 'request_id': 'req-1da9d945-e373-4c68-b8aa-9f8c248ae310'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:06.574300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '717a423af77843ba98ae813b6d469de3', 'request_id': 'req-1da9d945-e373-4c68-b8aa-9f8c248ae310'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:06.653133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 850/1706] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:06 2022] POST /identity/v3/projects => generated 381 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:06.654291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2be9e-e766-4d86-b3c0-c42c59cce3bd admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.702638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '717a423af77843ba98ae813b6d469de3', 'request_id': 'req-1da9d945-e373-4c68-b8aa-9f8c248ae310'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:06.722968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] The token cache is being invalidate because project 717a423af77843ba98ae813b6d469de3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:06.723285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:06.733143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:06.736773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 857/1707] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:06 2022] DELETE /identity/v3/projects/717a423af77843ba98ae813b6d469de3 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:06.739724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.742878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:06.903950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2be9e-e766-4d86-b3c0-c42c59cce3bd admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.921885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc6063-41fc-43cc-9641-35a6ee97711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.922622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc6063-41fc-43cc-9641-35a6ee97711e admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.922976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc6063-41fc-43cc-9641-35a6ee97711e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.923324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc6063-41fc-43cc-9641-35a6ee97711e admin admin] PATH_INFO: `/v3/projects/91172553622a4829948be99ea0295882/users/50d4e8a819084459bfc17293e4907880/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.934135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9d945-e373-4c68-b8aa-9f8c248ae310 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.942474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a162b0-97a7-4f4f-99b9-99e448d4393b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.945683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a162b0-97a7-4f4f-99b9-99e448d4393b admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.945683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a162b0-97a7-4f4f-99b9-99e448d4393b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.945683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a162b0-97a7-4f4f-99b9-99e448d4393b admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.945683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a162b0-97a7-4f4f-99b9-99e448d4393b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.947466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a162b0-97a7-4f4f-99b9-99e448d4393b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.967267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc6063-41fc-43cc-9641-35a6ee97711e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91172553622a4829948be99ea0295882, user_id=50d4e8a819084459bfc17293e4907880, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.970120 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc6063-41fc-43cc-9641-35a6ee97711e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:07.082243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 858/1708] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:06 2022] POST /identity/v3/users => generated 300 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:07.083716 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 851/1709] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:06 2022] PUT /identity/v3/projects/91172553622a4829948be99ea0295882/users/50d4e8a819084459bfc17293e4907880/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:07.084677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc6063-41fc-43cc-9641-35a6ee97711e admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:07.087248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a2556-bcca-4e2f-96ab-17202c058188 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:07.087563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a2556-bcca-4e2f-96ab-17202c058188 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:07.091205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a2556-bcca-4e2f-96ab-17202c058188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:07.091331 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e6fc6063-41fc-43cc-9641-35a6ee97711e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:07.156704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3c6a2556-bcca-4e2f-96ab-17202c058188 None None] MFA Rules not processed for user `71c5ac35123e425e9753f372f9c6bac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:07.158132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3c6a2556-bcca-4e2f-96ab-17202c058188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:07.272012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc6063-41fc-43cc-9641-35a6ee97711e admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:07.294054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab0cbc-5cd5-414f-b268-51deb2065ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:07.294928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab0cbc-5cd5-414f-b268-51deb2065ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:07.295213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab0cbc-5cd5-414f-b268-51deb2065ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:07.295489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab0cbc-5cd5-414f-b268-51deb2065ad7 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:07.296143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab0cbc-5cd5-414f-b268-51deb2065ad7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:07.303014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab0cbc-5cd5-414f-b268-51deb2065ad7 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:07.389195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 859/1710] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:07.392973 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b86c2-a8b0-4070-86a1-d84a975c7dcc None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:07.394896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b86c2-a8b0-4070-86a1-d84a975c7dcc None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:07.396652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b86c2-a8b0-4070-86a1-d84a975c7dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:07.414865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 852/1711] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:07 2022] POST /identity/v3/users => generated 297 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:07.416889 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab0cbc-5cd5-414f-b268-51deb2065ad7 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:07.453711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab0cbc-5cd5-414f-b268-51deb2065ad7 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:07.494574 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46924f8d-a903-40e3-ba21-2119305d0dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:07.495132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46924f8d-a903-40e3-ba21-2119305d0dcd admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:07.495414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46924f8d-a903-40e3-ba21-2119305d0dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:07.504915 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46924f8d-a903-40e3-ba21-2119305d0dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:07.505805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46924f8d-a903-40e3-ba21-2119305d0dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:07.520786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3e6b86c2-a8b0-4070-86a1-d84a975c7dcc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:07.523225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46924f8d-a903-40e3-ba21-2119305d0dcd admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:07.525948 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3e6b86c2-a8b0-4070-86a1-d84a975c7dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:07.564940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 860/1712] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:07.566927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05baed-55f3-4463-a498-063d0eca3465 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:07.567821 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05baed-55f3-4463-a498-063d0eca3465 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:07.568490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05baed-55f3-4463-a498-063d0eca3465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:07.577570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 853/1713] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:07.578726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46924f8d-a903-40e3-ba21-2119305d0dcd admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:07.582441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-46924f8d-a903-40e3-ba21-2119305d0dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:07.661413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-9b05baed-55f3-4463-a498-063d0eca3465 None None] MFA Rules not processed for user `50d4e8a819084459bfc17293e4907880`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:07.662665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9b05baed-55f3-4463-a498-063d0eca3465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:07.780406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46924f8d-a903-40e3-ba21-2119305d0dcd admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:07.797997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac238f20-43bb-4785-9998-c1cb8e295b40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:07.798638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac238f20-43bb-4785-9998-c1cb8e295b40 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:07.798927 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac238f20-43bb-4785-9998-c1cb8e295b40 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:07.799207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac238f20-43bb-4785-9998-c1cb8e295b40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:07.839113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac238f20-43bb-4785-9998-c1cb8e295b40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:07.844189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac238f20-43bb-4785-9998-c1cb8e295b40 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:07.870293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 861/1714] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:07.879556 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05baed-55f3-4463-a498-063d0eca3465 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:07.905841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 854/1715] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:07.911302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168fe8c-fd15-49ab-bd7a-8e81f5a20b7a None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:07.911594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168fe8c-fd15-49ab-bd7a-8e81f5a20b7a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:07.911876 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168fe8c-fd15-49ab-bd7a-8e81f5a20b7a None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:07.924399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 855/1716] 10.222.0.30 () {54 vars in 961 bytes} [Wed Aug 24 10:17:07 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:17:07.927519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0d360-423f-459d-8c05-037b2db89dbb None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:07.927736 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0d360-423f-459d-8c05-037b2db89dbb None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:07.928323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0d360-423f-459d-8c05-037b2db89dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:07.946411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05baed-55f3-4463-a498-063d0eca3465 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:07.975039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396af162-de81-465e-ae17-a7b3a559f952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:07.986764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396af162-de81-465e-ae17-a7b3a559f952 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:07.987088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396af162-de81-465e-ae17-a7b3a559f952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:07.987384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396af162-de81-465e-ae17-a7b3a559f952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:07.988529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396af162-de81-465e-ae17-a7b3a559f952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:07.990447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396af162-de81-465e-ae17-a7b3a559f952 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:08.051479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-16f0d360-423f-459d-8c05-037b2db89dbb None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:08.054769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-16f0d360-423f-459d-8c05-037b2db89dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:08.068497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 862/1717] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:08.070530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396af162-de81-465e-ae17-a7b3a559f952 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:08.080893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 856/1718] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:08.084967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3d646-4c73-4a2e-8551-5eef70881165 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:08.092879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3d646-4c73-4a2e-8551-5eef70881165 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:08.104621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3d646-4c73-4a2e-8551-5eef70881165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:08.112742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396af162-de81-465e-ae17-a7b3a559f952 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:08.144737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593653b4-6a3a-4c2c-8496-b9d9d6b097dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:08.156755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593653b4-6a3a-4c2c-8496-b9d9d6b097dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:08.157196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593653b4-6a3a-4c2c-8496-b9d9d6b097dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:08.157487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593653b4-6a3a-4c2c-8496-b9d9d6b097dc admin admin] PATH_INFO: `/v3/projects/7e908e1e79a241b788cc3569b0f71f59/users/6fc32f2c0aae42368e4dae2350283846/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:08.167467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-ace3d646-4c73-4a2e-8551-5eef70881165 None None] MFA Rules not processed for user `388174c87a2245ca89a634f3b177e166`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:08.171590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ace3d646-4c73-4a2e-8551-5eef70881165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:08.199132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593653b4-6a3a-4c2c-8496-b9d9d6b097dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e908e1e79a241b788cc3569b0f71f59, user_id=6fc32f2c0aae42368e4dae2350283846, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:08.206270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593653b4-6a3a-4c2c-8496-b9d9d6b097dc admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:08.337026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 857/1719] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:08.343681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3d646-4c73-4a2e-8551-5eef70881165 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:08.345431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ace3d646-4c73-4a2e-8551-5eef70881165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:08.372010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3d646-4c73-4a2e-8551-5eef70881165 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:08.372343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:08.380613 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 863/1720] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:08 2022] PUT /identity/v3/projects/7e908e1e79a241b788cc3569b0f71f59/users/6fc32f2c0aae42368e4dae2350283846/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:08.381935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb2106-3c16-4574-97a3-f67f57dfc803 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:08.382417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb2106-3c16-4574-97a3-f67f57dfc803 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:08.382900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb2106-3c16-4574-97a3-f67f57dfc803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:08.467595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-11bb2106-3c16-4574-97a3-f67f57dfc803 None None] MFA Rules not processed for user `b1aa47a883604336a8ab90d324470477`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:08.468841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-11bb2106-3c16-4574-97a3-f67f57dfc803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:08.483795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:08.505188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25373f14-4ab0-469d-b385-49b2cda70240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:08.505892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25373f14-4ab0-469d-b385-49b2cda70240 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:08.506793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25373f14-4ab0-469d-b385-49b2cda70240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:08.507182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25373f14-4ab0-469d-b385-49b2cda70240 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:08.515496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25373f14-4ab0-469d-b385-49b2cda70240 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:08.517690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25373f14-4ab0-469d-b385-49b2cda70240 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:08.683174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 864/1721] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Aug 24 10:17:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:08.683828 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb2106-3c16-4574-97a3-f67f57dfc803 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:08.702074 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb2106-3c16-4574-97a3-f67f57dfc803 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:08.718405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff59414-47e4-4d57-bb60-687a561f3f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:08.719412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff59414-47e4-4d57-bb60-687a561f3f9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:08.719864 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff59414-47e4-4d57-bb60-687a561f3f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:08.720365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff59414-47e4-4d57-bb60-687a561f3f9b admin admin] PATH_INFO: `/v3/projects/e43745a174ae4fd6b86af2f1a66a3b7f/users/4219b52ccf3b43d795fc94fcc8067b9d/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:08.742062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff59414-47e4-4d57-bb60-687a561f3f9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43745a174ae4fd6b86af2f1a66a3b7f, user_id=4219b52ccf3b43d795fc94fcc8067b9d, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:08.746703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff59414-47e4-4d57-bb60-687a561f3f9b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:08.751166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 858/1722] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:08 2022] POST /identity/v3/projects => generated 361 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:08.752718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25373f14-4ab0-469d-b385-49b2cda70240 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:08.903543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 865/1723] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:08 2022] PUT /identity/v3/projects/e43745a174ae4fd6b86af2f1a66a3b7f/users/4219b52ccf3b43d795fc94fcc8067b9d/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:08.905326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e007b72-c3ff-42bf-8720-49bbc159106c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:08.905857 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e007b72-c3ff-42bf-8720-49bbc159106c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:08.906383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e007b72-c3ff-42bf-8720-49bbc159106c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:08.917271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25373f14-4ab0-469d-b385-49b2cda70240 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:09.001468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-6e007b72-c3ff-42bf-8720-49bbc159106c None None] MFA Rules not processed for user `669c94a1591f41ae81d27f608987850f`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:09.010397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6e007b72-c3ff-42bf-8720-49bbc159106c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:09.167218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f773d-0852-40b3-852c-4c60816d404f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:09.171710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f773d-0852-40b3-852c-4c60816d404f admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.172210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f773d-0852-40b3-852c-4c60816d404f admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.172702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f773d-0852-40b3-852c-4c60816d404f admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.175615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f773d-0852-40b3-852c-4c60816d404f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:09.178890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f773d-0852-40b3-852c-4c60816d404f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:09.187046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f773d-0852-40b3-852c-4c60816d404f admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:09.249398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 859/1724] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 497 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:09.250907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f773d-0852-40b3-852c-4c60816d404f admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:09.288654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f773d-0852-40b3-852c-4c60816d404f admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:09.310888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10ed38-c1f2-4e73-b4bf-e6a2f5b873aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:09.311793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ed38-c1f2-4e73-b4bf-e6a2f5b873aa admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.312208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ed38-c1f2-4e73-b4bf-e6a2f5b873aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.312643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ed38-c1f2-4e73-b4bf-e6a2f5b873aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.313754 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10ed38-c1f2-4e73-b4bf-e6a2f5b873aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:09.316414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10ed38-c1f2-4e73-b4bf-e6a2f5b873aa admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:09.329045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 866/1725] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:09.342710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e007b72-c3ff-42bf-8720-49bbc159106c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:09.366499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 860/1726] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:09.374055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10ed38-c1f2-4e73-b4bf-e6a2f5b873aa admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:09.398549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10ed38-c1f2-4e73-b4bf-e6a2f5b873aa admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:09.422028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7989b8-94f0-47b9-aee7-825dbaeaae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:09.423624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7989b8-94f0-47b9-aee7-825dbaeaae48 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.423624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7989b8-94f0-47b9-aee7-825dbaeaae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.423791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7989b8-94f0-47b9-aee7-825dbaeaae48 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.424630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7989b8-94f0-47b9-aee7-825dbaeaae48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:09.427571 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7989b8-94f0-47b9-aee7-825dbaeaae48 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:09.504606 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e007b72-c3ff-42bf-8720-49bbc159106c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:09.512733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7a0ed-0d70-4b4e-b478-4830731170c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:09.513780 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7a0ed-0d70-4b4e-b478-4830731170c4 service placement] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.514268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7a0ed-0d70-4b4e-b478-4830731170c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.514749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7a0ed-0d70-4b4e-b478-4830731170c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.516900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-29e7a0ed-0d70-4b4e-b478-4830731170c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:09.529091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 861/1727] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:09 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:09.529091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7989b8-94f0-47b9-aee7-825dbaeaae48 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:09.533977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7a0ed-0d70-4b4e-b478-4830731170c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:09.536328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7a0ed-0d70-4b4e-b478-4830731170c4 service placement] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:09.550394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7989b8-94f0-47b9-aee7-825dbaeaae48 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:09.550991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 867/1728] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:09.552688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ec1a5-5aaa-4276-a1b8-02813bcde58a None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.553674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ec1a5-5aaa-4276-a1b8-02813bcde58a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.554105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ec1a5-5aaa-4276-a1b8-02813bcde58a None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.556213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 868/1729] 10.222.0.30 () {54 vars in 912 bytes} [Wed Aug 24 10:17:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:17:09.557901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ec1a5-5aaa-4276-a1b8-02813bcde58a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:09.567195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9d3c2-b544-4b18-8028-d1e7d5a832d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:09.568065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d3c2-b544-4b18-8028-d1e7d5a832d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.568490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d3c2-b544-4b18-8028-d1e7d5a832d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.568926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d3c2-b544-4b18-8028-d1e7d5a832d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.569742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9d3c2-b544-4b18-8028-d1e7d5a832d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:09.571660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9d3c2-b544-4b18-8028-d1e7d5a832d9 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:09.574884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ec1a5-5aaa-4276-a1b8-02813bcde58a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:09.581283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d03453-979e-406c-bd76-bf225d74ba4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:09.583392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d03453-979e-406c-bd76-bf225d74ba4c admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.583392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d03453-979e-406c-bd76-bf225d74ba4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.583392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d03453-979e-406c-bd76-bf225d74ba4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.584255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d03453-979e-406c-bd76-bf225d74ba4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:09.586525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d03453-979e-406c-bd76-bf225d74ba4c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:09.600791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 862/1730] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:09.602155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a879c0-505b-42af-9941-3f2247539273 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.602443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a879c0-505b-42af-9941-3f2247539273 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.602720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a879c0-505b-42af-9941-3f2247539273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.629011 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-87a879c0-505b-42af-9941-3f2247539273 None None] MFA Rules not processed for user `669c94a1591f41ae81d27f608987850f`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:09.631013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-87a879c0-505b-42af-9941-3f2247539273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:09.644575 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 863/1731] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:09.645521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a879c0-505b-42af-9941-3f2247539273 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:09.668228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a879c0-505b-42af-9941-3f2247539273 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:09.671483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f60ce-5bfb-4362-a2a1-f1b914729600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:09.673322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f60ce-5bfb-4362-a2a1-f1b914729600 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.673820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f60ce-5bfb-4362-a2a1-f1b914729600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.674839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f60ce-5bfb-4362-a2a1-f1b914729600 admin admin] PATH_INFO: `/v3/projects/7e908e1e79a241b788cc3569b0f71f59/users/6fc32f2c0aae42368e4dae2350283846/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.677618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f60ce-5bfb-4362-a2a1-f1b914729600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e908e1e79a241b788cc3569b0f71f59, user_id=6fc32f2c0aae42368e4dae2350283846, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:09.681987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f60ce-5bfb-4362-a2a1-f1b914729600 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:09.699076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 864/1732] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:09 2022] PUT /identity/v3/projects/7e908e1e79a241b788cc3569b0f71f59/users/6fc32f2c0aae42368e4dae2350283846/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:09.700443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f60ce-5bfb-4362-a2a1-f1b914729600 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:09.707627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 869/1733] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:09 2022] POST /identity/v3/projects => generated 409 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:09.709348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4651fa-cd1b-4dbd-9f5d-e279da327b83 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.710205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4651fa-cd1b-4dbd-9f5d-e279da327b83 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.713764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4651fa-cd1b-4dbd-9f5d-e279da327b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.759071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ce4651fa-cd1b-4dbd-9f5d-e279da327b83 None None] MFA Rules not processed for user `ae9b6525009b41288dd918437acee92a`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:09.761191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ce4651fa-cd1b-4dbd-9f5d-e279da327b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:09.819670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f60ce-5bfb-4362-a2a1-f1b914729600 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:09.855222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 870/1734] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:17:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:09.867670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4651fa-cd1b-4dbd-9f5d-e279da327b83 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:09.927058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78830de9-85a7-489c-b770-20901529e2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:09.928282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78830de9-85a7-489c-b770-20901529e2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.928569 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78830de9-85a7-489c-b770-20901529e2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.928850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78830de9-85a7-489c-b770-20901529e2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.930123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78830de9-85a7-489c-b770-20901529e2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:09.932289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78830de9-85a7-489c-b770-20901529e2e4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:09.948759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 865/1735] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:09.950068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78830de9-85a7-489c-b770-20901529e2e4 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.010687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4651fa-cd1b-4dbd-9f5d-e279da327b83 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.026966 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c167a-a338-420f-b8ad-809edb274a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.028661 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c167a-a338-420f-b8ad-809edb274a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.028968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c167a-a338-420f-b8ad-809edb274a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.029246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c167a-a338-420f-b8ad-809edb274a6a admin admin] PATH_INFO: `/v3/projects/e43745a174ae4fd6b86af2f1a66a3b7f/users/4219b52ccf3b43d795fc94fcc8067b9d/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.033257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c167a-a338-420f-b8ad-809edb274a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43745a174ae4fd6b86af2f1a66a3b7f, user_id=4219b52ccf3b43d795fc94fcc8067b9d, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.043012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c167a-a338-420f-b8ad-809edb274a6a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.064324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 871/1736] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:09 2022] PUT /identity/v3/projects/e43745a174ae4fd6b86af2f1a66a3b7f/users/4219b52ccf3b43d795fc94fcc8067b9d/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:10.078809 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868214a0-5624-4812-9df8-fad5d78f0340 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.079097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868214a0-5624-4812-9df8-fad5d78f0340 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.079359 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868214a0-5624-4812-9df8-fad5d78f0340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.123480 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-868214a0-5624-4812-9df8-fad5d78f0340 None None] MFA Rules not processed for user `6fc32f2c0aae42368e4dae2350283846`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:10.124586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78830de9-85a7-489c-b770-20901529e2e4 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.126587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-868214a0-5624-4812-9df8-fad5d78f0340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:10.207494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b49eb5-53ac-4dae-b332-61b22b1215cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.208505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b49eb5-53ac-4dae-b332-61b22b1215cb service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.209085 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b49eb5-53ac-4dae-b332-61b22b1215cb service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.210831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b49eb5-53ac-4dae-b332-61b22b1215cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.270600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 872/1737] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:10.271178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868214a0-5624-4812-9df8-fad5d78f0340 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.307758 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b49eb5-53ac-4dae-b332-61b22b1215cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.309885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b49eb5-53ac-4dae-b332-61b22b1215cb service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.321325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 866/1738] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:10.322483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b49eb5-53ac-4dae-b332-61b22b1215cb service nova] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.330430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d5b49eb5-53ac-4dae-b332-61b22b1215cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:10.357885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868214a0-5624-4812-9df8-fad5d78f0340 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.364071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1da090-8cb2-43c9-8716-ac810abef2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.364791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1da090-8cb2-43c9-8716-ac810abef2dc admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.365764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1da090-8cb2-43c9-8716-ac810abef2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.366119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1da090-8cb2-43c9-8716-ac810abef2dc admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.367103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1da090-8cb2-43c9-8716-ac810abef2dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.370268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1da090-8cb2-43c9-8716-ac810abef2dc admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.482643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b49eb5-53ac-4dae-b332-61b22b1215cb service nova] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.488816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688ae60-a307-4d77-afe5-68a79edc55a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.489637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688ae60-a307-4d77-afe5-68a79edc55a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.489994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688ae60-a307-4d77-afe5-68a79edc55a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.490428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688ae60-a307-4d77-afe5-68a79edc55a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.499658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688ae60-a307-4d77-afe5-68a79edc55a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.501278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688ae60-a307-4d77-afe5-68a79edc55a1 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.511212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 867/1739] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:10.512622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688ae60-a307-4d77-afe5-68a79edc55a1 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.525925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688ae60-a307-4d77-afe5-68a79edc55a1 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.531335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685f251-780a-4643-ac35-f908381cd56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.532497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685f251-780a-4643-ac35-f908381cd56a admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.532689 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685f251-780a-4643-ac35-f908381cd56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.534083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685f251-780a-4643-ac35-f908381cd56a admin admin] PATH_INFO: `/v3/projects/a287ced976894d63bb9c31d833362bec/users/b37e1dc507534091a058031cee67c8d7/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.551481 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 873/1740] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:10 2022] POST /identity/v3/users => generated 305 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:10.553150 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d3600-56e5-473e-a4ef-2f89c623ab10 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.553451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685f251-780a-4643-ac35-f908381cd56a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a287ced976894d63bb9c31d833362bec, user_id=b37e1dc507534091a058031cee67c8d7, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.556452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d3600-56e5-473e-a4ef-2f89c623ab10 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.556609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685f251-780a-4643-ac35-f908381cd56a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.557088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d3600-56e5-473e-a4ef-2f89c623ab10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.620313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-d61d3600-56e5-473e-a4ef-2f89c623ab10 None None] MFA Rules not processed for user `4219b52ccf3b43d795fc94fcc8067b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:10.621890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d61d3600-56e5-473e-a4ef-2f89c623ab10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:10.680370 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 868/1741] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:10 2022] PUT /identity/v3/projects/a287ced976894d63bb9c31d833362bec/users/b37e1dc507534091a058031cee67c8d7/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:10.682730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685f251-780a-4643-ac35-f908381cd56a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.686108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1685f251-780a-4643-ac35-f908381cd56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:10.736670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 874/1742] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:10.737784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d3600-56e5-473e-a4ef-2f89c623ab10 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.760995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d3600-56e5-473e-a4ef-2f89c623ab10 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.769010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:10.780390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685f251-780a-4643-ac35-f908381cd56a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.795708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4adfd4-dadb-41d5-b5e5-94277f285cbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.796759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4adfd4-dadb-41d5-b5e5-94277f285cbd service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.797077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4adfd4-dadb-41d5-b5e5-94277f285cbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.797719 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4adfd4-dadb-41d5-b5e5-94277f285cbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.848845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c135-8d88-42f2-9937-48a21b0e8ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.849995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c135-8d88-42f2-9937-48a21b0e8ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.850554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c135-8d88-42f2-9937-48a21b0e8ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.851031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c135-8d88-42f2-9937-48a21b0e8ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.851957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010c135-8d88-42f2-9937-48a21b0e8ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.854603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010c135-8d88-42f2-9937-48a21b0e8ec6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.867222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 875/1743] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:10.868794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c135-8d88-42f2-9937-48a21b0e8ec6 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.870206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4adfd4-dadb-41d5-b5e5-94277f285cbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.871945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4adfd4-dadb-41d5-b5e5-94277f285cbd service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.880991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c135-8d88-42f2-9937-48a21b0e8ec6 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.883487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 869/1744] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:10.885212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4adfd4-dadb-41d5-b5e5-94277f285cbd service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.886753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a271c9-e957-445f-b3ee-79a8c5b6a6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.886882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ac4adfd4-dadb-41d5-b5e5-94277f285cbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:10.887476 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a271c9-e957-445f-b3ee-79a8c5b6a6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.887828 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a271c9-e957-445f-b3ee-79a8c5b6a6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.888176 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a271c9-e957-445f-b3ee-79a8c5b6a6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.889099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a271c9-e957-445f-b3ee-79a8c5b6a6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.890975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a271c9-e957-445f-b3ee-79a8c5b6a6c2 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.902285 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4adfd4-dadb-41d5-b5e5-94277f285cbd service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.906520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66295f7-de32-4df7-968b-a9d801bd7222 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.907128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 876/1745] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:10.907654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66295f7-de32-4df7-968b-a9d801bd7222 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.908038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66295f7-de32-4df7-968b-a9d801bd7222 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.908145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a271c9-e957-445f-b3ee-79a8c5b6a6c2 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.908714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66295f7-de32-4df7-968b-a9d801bd7222 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.933537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a271c9-e957-445f-b3ee-79a8c5b6a6c2 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.948583 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6480354-0ba9-4f72-9bbc-f5fe19b855f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.949285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6480354-0ba9-4f72-9bbc-f5fe19b855f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.949570 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6480354-0ba9-4f72-9bbc-f5fe19b855f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.950054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6480354-0ba9-4f72-9bbc-f5fe19b855f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.950715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6480354-0ba9-4f72-9bbc-f5fe19b855f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.954619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6480354-0ba9-4f72-9bbc-f5fe19b855f5 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.987850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 877/1746] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:10.987850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6480354-0ba9-4f72-9bbc-f5fe19b855f5 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.998442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6480354-0ba9-4f72-9bbc-f5fe19b855f5 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.003712 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0f2ff-8c5b-41a0-90e5-7f8dfe914e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.004392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0f2ff-8c5b-41a0-90e5-7f8dfe914e26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.004676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0f2ff-8c5b-41a0-90e5-7f8dfe914e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.004989 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0f2ff-8c5b-41a0-90e5-7f8dfe914e26 admin admin] PATH_INFO: `/v3/projects/a287ced976894d63bb9c31d833362bec/users/b37e1dc507534091a058031cee67c8d7/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.007203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0f2ff-8c5b-41a0-90e5-7f8dfe914e26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a287ced976894d63bb9c31d833362bec, user_id=b37e1dc507534091a058031cee67c8d7, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.010626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0f2ff-8c5b-41a0-90e5-7f8dfe914e26 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.024706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 878/1747] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:10 2022] PUT /identity/v3/projects/a287ced976894d63bb9c31d833362bec/users/b37e1dc507534091a058031cee67c8d7/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:11.025894 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0f2ff-8c5b-41a0-90e5-7f8dfe914e26 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.076848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66295f7-de32-4df7-968b-a9d801bd7222 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.078383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66295f7-de32-4df7-968b-a9d801bd7222 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.107291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 870/1748] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:11.109570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66295f7-de32-4df7-968b-a9d801bd7222 service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.133044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66295f7-de32-4df7-968b-a9d801bd7222 service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.144716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0f2ff-8c5b-41a0-90e5-7f8dfe914e26 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.149086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c617e-b71e-4742-857b-31b21c11c45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.150000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c617e-b71e-4742-857b-31b21c11c45a admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.150342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c617e-b71e-4742-857b-31b21c11c45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.150749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c617e-b71e-4742-857b-31b21c11c45a admin admin] PATH_INFO: `/v3/projects/58353a1c5ff744a99a3814223bdddee8/users/4851c4ac15784cba81d6b74166880e99/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.156124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.156802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.157233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.157626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] PATH_INFO: `/v3/users/f6123c5c34a5457bb6c1ae34501bc20c` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.158965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6123c5c34a5457bb6c1ae34501bc20c)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.161979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.170784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c617e-b71e-4742-857b-31b21c11c45a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58353a1c5ff744a99a3814223bdddee8, user_id=4851c4ac15784cba81d6b74166880e99, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.173902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c617e-b71e-4742-857b-31b21c11c45a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.390565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 871/1749] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:11 2022] PUT /identity/v3/projects/58353a1c5ff744a99a3814223bdddee8/users/4851c4ac15784cba81d6b74166880e99/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:11.393082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346aec4-066a-4950-b974-5ae1ad66445d None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.393554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346aec4-066a-4950-b974-5ae1ad66445d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.394116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346aec4-066a-4950-b974-5ae1ad66445d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.462848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-9346aec4-066a-4950-b974-5ae1ad66445d None None] MFA Rules not processed for user `b37e1dc507534091a058031cee67c8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:11.464396 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9346aec4-066a-4950-b974-5ae1ad66445d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:11.533093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6123c5c34a5457bb6c1ae34501bc20c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:11.534120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6123c5c34a5457bb6c1ae34501bc20c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:11.553804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6123c5c34a5457bb6c1ae34501bc20c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:11.605368 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6123c5c34a5457bb6c1ae34501bc20c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:11.660589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 872/1750] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:11 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:11.661937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d9005-511d-4d55-a17b-52b1cd335a5f None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.662223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d9005-511d-4d55-a17b-52b1cd335a5f None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.663686 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d9005-511d-4d55-a17b-52b1cd335a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.747196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-b31d9005-511d-4d55-a17b-52b1cd335a5f None None] MFA Rules not processed for user `4851c4ac15784cba81d6b74166880e99`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:11.747500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b31d9005-511d-4d55-a17b-52b1cd335a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:11.749505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6123c5c34a5457bb6c1ae34501bc20c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:11.770416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 879/1751] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:11 2022] DELETE /identity/v3/users/f6123c5c34a5457bb6c1ae34501bc20c => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:11.771092 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.773996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:11.891854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 873/1752] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:11 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:11.892984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d9005-511d-4d55-a17b-52b1cd335a5f None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.901770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b31d9005-511d-4d55-a17b-52b1cd335a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:11.912679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3491d8c-2af1-49a3-9852-fe7ca7d198ce admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.919510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:11.929567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad34dd6-a60e-43d7-a3fe-fd9f50f44a74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.930509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34dd6-a60e-43d7-a3fe-fd9f50f44a74 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.930509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34dd6-a60e-43d7-a3fe-fd9f50f44a74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.930734 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34dd6-a60e-43d7-a3fe-fd9f50f44a74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.933530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9ad34dd6-a60e-43d7-a3fe-fd9f50f44a74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:12.118374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d9005-511d-4d55-a17b-52b1cd335a5f None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:12.126861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86664e-1251-4935-8aa7-c3767066259b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:12.127569 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86664e-1251-4935-8aa7-c3767066259b service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.127943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86664e-1251-4935-8aa7-c3767066259b service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.128290 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86664e-1251-4935-8aa7-c3767066259b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.130770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad34dd6-a60e-43d7-a3fe-fd9f50f44a74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:12.131154 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6b86664e-1251-4935-8aa7-c3767066259b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:12.132240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad34dd6-a60e-43d7-a3fe-fd9f50f44a74 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:12.182656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 880/1753] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:12.184566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec06226-1280-4b4c-9c3c-068271f98a9c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.184889 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec06226-1280-4b4c-9c3c-068271f98a9c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.185163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec06226-1280-4b4c-9c3c-068271f98a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.235755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-5ec06226-1280-4b4c-9c3c-068271f98a9c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:12.239947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5ec06226-1280-4b4c-9c3c-068271f98a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:12.337736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 881/1754] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:12.341791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec06226-1280-4b4c-9c3c-068271f98a9c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:12.344986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5ec06226-1280-4b4c-9c3c-068271f98a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:12.345480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86664e-1251-4935-8aa7-c3767066259b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:12.347299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86664e-1251-4935-8aa7-c3767066259b service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:12.360569 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 874/1755] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:11 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:12.362347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e867668-5d14-4922-b3f9-770992f9b399 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.362847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e867668-5d14-4922-b3f9-770992f9b399 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.363361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e867668-5d14-4922-b3f9-770992f9b399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.383827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec06226-1280-4b4c-9c3c-068271f98a9c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:12.423755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003fb43-a21d-4ea1-9aed-2bc5d91f9c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:12.424394 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003fb43-a21d-4ea1-9aed-2bc5d91f9c15 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.424681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003fb43-a21d-4ea1-9aed-2bc5d91f9c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.424983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003fb43-a21d-4ea1-9aed-2bc5d91f9c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.431799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003fb43-a21d-4ea1-9aed-2bc5d91f9c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:12.433130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-0e867668-5d14-4922-b3f9-770992f9b399 None None] MFA Rules not processed for user `c94c65ae14e94796a3b3bd0bb29a91b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:12.434483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0e867668-5d14-4922-b3f9-770992f9b399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:12.436529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003fb43-a21d-4ea1-9aed-2bc5d91f9c15 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:12.556223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 875/1756] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:12.556223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e867668-5d14-4922-b3f9-770992f9b399 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:12.565818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0e867668-5d14-4922-b3f9-770992f9b399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:12.636800 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 882/1757] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:12 2022] POST /identity/v3/projects => generated 407 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:12.641519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003fb43-a21d-4ea1-9aed-2bc5d91f9c15 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:12.716560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e867668-5d14-4922-b3f9-770992f9b399 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:12.945560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249519a6-8a53-4033-bf8b-43ffbe0abfd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:12.946613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249519a6-8a53-4033-bf8b-43ffbe0abfd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.946901 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249519a6-8a53-4033-bf8b-43ffbe0abfd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.947167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249519a6-8a53-4033-bf8b-43ffbe0abfd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.955313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003fb43-a21d-4ea1-9aed-2bc5d91f9c15 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:12.985195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfa30e-f3c8-48b2-816b-87e82c6a05c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:12.986819 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249519a6-8a53-4033-bf8b-43ffbe0abfd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:12.989596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249519a6-8a53-4033-bf8b-43ffbe0abfd0 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:12.990288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfa30e-f3c8-48b2-816b-87e82c6a05c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.991897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfa30e-f3c8-48b2-816b-87e82c6a05c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.992296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfa30e-f3c8-48b2-816b-87e82c6a05c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.994508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfa30e-f3c8-48b2-816b-87e82c6a05c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:12.998509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfa30e-f3c8-48b2-816b-87e82c6a05c7 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:13.023664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 876/1758] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:13.024513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249519a6-8a53-4033-bf8b-43ffbe0abfd0 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:13.209528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 883/1759] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:12 2022] POST /identity/v3/users => generated 310 bytes in 569 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:13.215221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfa30e-f3c8-48b2-816b-87e82c6a05c7 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:13.229841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:13.253423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249519a6-8a53-4033-bf8b-43ffbe0abfd0 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:13.270558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad2aeb-23c1-4146-af1b-4a140ea5032e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:13.272001 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad2aeb-23c1-4146-af1b-4a140ea5032e service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:13.272298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad2aeb-23c1-4146-af1b-4a140ea5032e service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:13.272578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad2aeb-23c1-4146-af1b-4a140ea5032e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:13.277509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9bad2aeb-23c1-4146-af1b-4a140ea5032e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:13.289822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfa30e-f3c8-48b2-816b-87e82c6a05c7 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:13.317647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54a28e-d24d-4021-a392-5232af14f1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:13.318579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54a28e-d24d-4021-a392-5232af14f1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:13.318865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54a28e-d24d-4021-a392-5232af14f1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:13.319140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54a28e-d24d-4021-a392-5232af14f1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:13.321960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54a28e-d24d-4021-a392-5232af14f1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:13.325314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54a28e-d24d-4021-a392-5232af14f1e0 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:13.399113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 884/1760] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:13.400641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54a28e-d24d-4021-a392-5232af14f1e0 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:13.408551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad2aeb-23c1-4146-af1b-4a140ea5032e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:13.410201 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad2aeb-23c1-4146-af1b-4a140ea5032e service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:13.432447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 877/1761] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:13.456705 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54a28e-d24d-4021-a392-5232af14f1e0 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:13.497764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d7944-ee24-44a3-88f9-93e4162f6277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:13.498556 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7944-ee24-44a3-88f9-93e4162f6277 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:13.499028 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7944-ee24-44a3-88f9-93e4162f6277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:13.522315 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7944-ee24-44a3-88f9-93e4162f6277 admin admin] PATH_INFO: `/v3/projects/1ecac7aedfe142b880f8c72e2206bb13/users/c29f28b1c3da4ddb9cc497333555ad35/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:13.588109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d7944-ee24-44a3-88f9-93e4162f6277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ecac7aedfe142b880f8c72e2206bb13, user_id=c29f28b1c3da4ddb9cc497333555ad35, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:13.591942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d7944-ee24-44a3-88f9-93e4162f6277 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:13.729724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 885/1762] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:13 2022] PUT /identity/v3/projects/1ecac7aedfe142b880f8c72e2206bb13/users/c29f28b1c3da4ddb9cc497333555ad35/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:13.730719 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad2aeb-23c1-4146-af1b-4a140ea5032e service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:13.744698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad2aeb-23c1-4146-af1b-4a140ea5032e service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:13.875718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b94d5-57eb-43ec-9382-5eb2b640efea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:13.876645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b94d5-57eb-43ec-9382-5eb2b640efea admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:13.877121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b94d5-57eb-43ec-9382-5eb2b640efea admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:13.877670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b94d5-57eb-43ec-9382-5eb2b640efea admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:13.878649 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b94d5-57eb-43ec-9382-5eb2b640efea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:13.881182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b94d5-57eb-43ec-9382-5eb2b640efea admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:13.897513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 878/1763] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:13.902433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d7944-ee24-44a3-88f9-93e4162f6277 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:13.925575 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d7944-ee24-44a3-88f9-93e4162f6277 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:13.930602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c3784-8f8a-4fb3-822e-29867ae4c646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:13.931823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3784-8f8a-4fb3-822e-29867ae4c646 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:13.932100 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3784-8f8a-4fb3-822e-29867ae4c646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:13.932451 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3784-8f8a-4fb3-822e-29867ae4c646 admin admin] PATH_INFO: `/v3/projects/1ecac7aedfe142b880f8c72e2206bb13/users/c29f28b1c3da4ddb9cc497333555ad35/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:13.935564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c3784-8f8a-4fb3-822e-29867ae4c646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ecac7aedfe142b880f8c72e2206bb13, user_id=c29f28b1c3da4ddb9cc497333555ad35, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:13.940730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c3784-8f8a-4fb3-822e-29867ae4c646 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:13.964102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 886/1764] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:13 2022] PUT /identity/v3/projects/1ecac7aedfe142b880f8c72e2206bb13/users/c29f28b1c3da4ddb9cc497333555ad35/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:13.966130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bd05c-b73c-4db6-92c3-8513d19d2413 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:13.966593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bd05c-b73c-4db6-92c3-8513d19d2413 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:13.967033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bd05c-b73c-4db6-92c3-8513d19d2413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:13.998287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fcd00-dbcf-4588-8ece-0fa1953038e7 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:13.998858 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fcd00-dbcf-4588-8ece-0fa1953038e7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:14.000293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fcd00-dbcf-4588-8ece-0fa1953038e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:14.035719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0a0fcd00-dbcf-4588-8ece-0fa1953038e7 None None] MFA Rules not processed for user `71c5ac35123e425e9753f372f9c6bac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:14.037710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0a0fcd00-dbcf-4588-8ece-0fa1953038e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:14.062195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-ae0bd05c-b73c-4db6-92c3-8513d19d2413 None None] MFA Rules not processed for user `c29f28b1c3da4ddb9cc497333555ad35`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:14.063531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ae0bd05c-b73c-4db6-92c3-8513d19d2413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:14.220776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 887/1765] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:14.273115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 879/1766] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:14.285590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fcd00-dbcf-4588-8ece-0fa1953038e7 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:14.288435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:14.432744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fcd00-dbcf-4588-8ece-0fa1953038e7 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:14.438801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d400b-15ee-41fd-be35-ed59f0f81195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:14.440138 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d400b-15ee-41fd-be35-ed59f0f81195 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:14.440721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d400b-15ee-41fd-be35-ed59f0f81195 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:14.441188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d400b-15ee-41fd-be35-ed59f0f81195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:14.455484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d400b-15ee-41fd-be35-ed59f0f81195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:14.457296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d400b-15ee-41fd-be35-ed59f0f81195 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:14.482252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 888/1767] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:14 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:14.631047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bd05c-b73c-4db6-92c3-8513d19d2413 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:14.632955 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ae0bd05c-b73c-4db6-92c3-8513d19d2413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:14.907802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bd05c-b73c-4db6-92c3-8513d19d2413 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:14.920751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:14.922400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:14.923386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:14.923894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] PATH_INFO: `/v3/projects/5cf27cd5d989481fbfe3f0db7f95eefe` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:14.929171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cf27cd5d989481fbfe3f0db7f95eefe)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:14.931967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:15.165011 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cf27cd5d989481fbfe3f0db7f95eefe', 'request_id': 'req-8d5c7267-f04e-4429-8823-867681806853'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:15.173411 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cf27cd5d989481fbfe3f0db7f95eefe', 'request_id': 'req-8d5c7267-f04e-4429-8823-867681806853'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:15.184696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cf27cd5d989481fbfe3f0db7f95eefe', 'request_id': 'req-8d5c7267-f04e-4429-8823-867681806853'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:15.248195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d400b-15ee-41fd-be35-ed59f0f81195 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:15.256403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] The token cache is being invalidate because project 5cf27cd5d989481fbfe3f0db7f95eefe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:15.257091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:15.258131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:15.260634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 880/1768] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:14 2022] DELETE /identity/v3/projects/5cf27cd5d989481fbfe3f0db7f95eefe => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:15.265077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:15.268076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:15.281082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d400b-15ee-41fd-be35-ed59f0f81195 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:15.418826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c7267-f04e-4429-8823-867681806853 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:15.433482 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e4f507-6b4c-43ab-89d0-b7df14cacfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:15.434134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4f507-6b4c-43ab-89d0-b7df14cacfb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:15.434464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4f507-6b4c-43ab-89d0-b7df14cacfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:15.434748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4f507-6b4c-43ab-89d0-b7df14cacfb2 admin admin] PATH_INFO: `/v3/users/6fc32f2c0aae42368e4dae2350283846` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:15.442499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e4f507-6b4c-43ab-89d0-b7df14cacfb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fc32f2c0aae42368e4dae2350283846)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:15.444467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e4f507-6b4c-43ab-89d0-b7df14cacfb2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:15.458934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba153e-0686-4c4a-94ec-2a1fe968e850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:15.460037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba153e-0686-4c4a-94ec-2a1fe968e850 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:15.460415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba153e-0686-4c4a-94ec-2a1fe968e850 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:15.461344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba153e-0686-4c4a-94ec-2a1fe968e850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:15.470914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-64ba153e-0686-4c4a-94ec-2a1fe968e850 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:15.499641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:15.614686 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba153e-0686-4c4a-94ec-2a1fe968e850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:15.617862 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba153e-0686-4c4a-94ec-2a1fe968e850 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:15.634674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 889/1769] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:15 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:15.647836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98901ee-0906-494e-859d-b0e601d72e57 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:15.650815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98901ee-0906-494e-859d-b0e601d72e57 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:15.651213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98901ee-0906-494e-859d-b0e601d72e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:15.730794 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-20e4f507-6b4c-43ab-89d0-b7df14cacfb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fc32f2c0aae42368e4dae2350283846'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:15.732478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-20e4f507-6b4c-43ab-89d0-b7df14cacfb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fc32f2c0aae42368e4dae2350283846'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:15.745999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-20e4f507-6b4c-43ab-89d0-b7df14cacfb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fc32f2c0aae42368e4dae2350283846'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:15.747302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-20e4f507-6b4c-43ab-89d0-b7df14cacfb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fc32f2c0aae42368e4dae2350283846'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:15.790997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b98901ee-0906-494e-859d-b0e601d72e57 None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:15.792351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b98901ee-0906-494e-859d-b0e601d72e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:15.932543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 890/1770] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:16.025294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-20e4f507-6b4c-43ab-89d0-b7df14cacfb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fc32f2c0aae42368e4dae2350283846'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:16.051256 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 881/1771] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:15 2022] DELETE /identity/v3/users/6fc32f2c0aae42368e4dae2350283846 => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:16.526475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98901ee-0906-494e-859d-b0e601d72e57 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:16.527901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b98901ee-0906-494e-859d-b0e601d72e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:16.921881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98901ee-0906-494e-859d-b0e601d72e57 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:17.080146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4319722-a0ee-4090-a936-d51598bc282d None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:17.080657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4319722-a0ee-4090-a936-d51598bc282d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:17.081484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4319722-a0ee-4090-a936-d51598bc282d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:17.126776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a4319722-a0ee-4090-a936-d51598bc282d None None] MFA Rules not processed for user `ae9b6525009b41288dd918437acee92a`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:17.128375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a4319722-a0ee-4090-a936-d51598bc282d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:17.197251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:17.225237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b4dc2-e552-41c5-8698-177ab7ebfa65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:17.225878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b4dc2-e552-41c5-8698-177ab7ebfa65 service swift] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:17.226153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b4dc2-e552-41c5-8698-177ab7ebfa65 service swift] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:17.227261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b4dc2-e552-41c5-8698-177ab7ebfa65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:17.347058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 882/1772] 10.222.0.81 () {58 vars in 1015 bytes} [Wed Aug 24 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:17.348351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4319722-a0ee-4090-a936-d51598bc282d None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:17.355628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a4319722-a0ee-4090-a936-d51598bc282d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:17.481992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b4dc2-e552-41c5-8698-177ab7ebfa65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:17.483813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b4dc2-e552-41c5-8698-177ab7ebfa65 service swift] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:17.500432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 891/1773] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:17:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 970 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:17:17.500432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b4dc2-e552-41c5-8698-177ab7ebfa65 service swift] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:17.500432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b13b4dc2-e552-41c5-8698-177ab7ebfa65 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:17.512555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b4dc2-e552-41c5-8698-177ab7ebfa65 service swift] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:17.516444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ab48b-90c7-40a2-b01d-87780f72d2c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:17.517156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ab48b-90c7-40a2-b01d-87780f72d2c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:17.517442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ab48b-90c7-40a2-b01d-87780f72d2c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:17.517760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ab48b-90c7-40a2-b01d-87780f72d2c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:17.543578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ab48b-90c7-40a2-b01d-87780f72d2c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:17.545784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ab48b-90c7-40a2-b01d-87780f72d2c4 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:17.562047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 892/1774] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:17.579406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4319722-a0ee-4090-a936-d51598bc282d None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:17.585768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:17.586710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:17.587964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:17.588648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] PATH_INFO: `/v3/projects/7e908e1e79a241b788cc3569b0f71f59` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:17.590184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e908e1e79a241b788cc3569b0f71f59)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:17.593116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:17.858938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e908e1e79a241b788cc3569b0f71f59', 'request_id': 'req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:17.866491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e908e1e79a241b788cc3569b0f71f59', 'request_id': 'req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:17.895478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e908e1e79a241b788cc3569b0f71f59', 'request_id': 'req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:18.096419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] The token cache is being invalidate because project 7e908e1e79a241b788cc3569b0f71f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:18.099833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:18.101474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-406d07d9-57cc-4fb9-a8a1-5caa705a41e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:18.113630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 883/1775] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:17 2022] DELETE /identity/v3/projects/7e908e1e79a241b788cc3569b0f71f59 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:18.119369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95afc45-f15a-4f79-a914-8e440d543c51 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:18.120645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95afc45-f15a-4f79-a914-8e440d543c51 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:18.121084 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95afc45-f15a-4f79-a914-8e440d543c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:18.188260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b95afc45-f15a-4f79-a914-8e440d543c51 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:18.190250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b95afc45-f15a-4f79-a914-8e440d543c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:18.255497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:18.413547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 893/1776] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:18.417308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bef0ba-3909-41b4-a15a-77d191538746 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:18.417656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bef0ba-3909-41b4-a15a-77d191538746 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:18.417935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bef0ba-3909-41b4-a15a-77d191538746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:18.452194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-39bef0ba-3909-41b4-a15a-77d191538746 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:18.463246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-39bef0ba-3909-41b4-a15a-77d191538746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:18.486386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 884/1777] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:18.490633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95afc45-f15a-4f79-a914-8e440d543c51 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:18.507887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95afc45-f15a-4f79-a914-8e440d543c51 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:18.532918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515acfb-87e9-4487-b649-cc839ff4e530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:18.532918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515acfb-87e9-4487-b649-cc839ff4e530 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:18.532918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515acfb-87e9-4487-b649-cc839ff4e530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:18.533341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515acfb-87e9-4487-b649-cc839ff4e530 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:18.533694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515acfb-87e9-4487-b649-cc839ff4e530 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:18.533994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515acfb-87e9-4487-b649-cc839ff4e530 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:18.538449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515acfb-87e9-4487-b649-cc839ff4e530 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:18.588253 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 894/1778] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:18.589301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bef0ba-3909-41b4-a15a-77d191538746 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:18.623017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bef0ba-3909-41b4-a15a-77d191538746 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:18.666587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840aa78a-51ec-4843-853a-014c487fd2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:18.668638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840aa78a-51ec-4843-853a-014c487fd2d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:18.668944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840aa78a-51ec-4843-853a-014c487fd2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:18.672696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840aa78a-51ec-4843-853a-014c487fd2d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:18.674569 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840aa78a-51ec-4843-853a-014c487fd2d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:18.677132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840aa78a-51ec-4843-853a-014c487fd2d5 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:18.862777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 885/1779] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:18 2022] POST /identity/v3/projects => generated 385 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:18.878377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515acfb-87e9-4487-b649-cc839ff4e530 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:18.912986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515acfb-87e9-4487-b649-cc839ff4e530 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:19.115265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065d274-621c-445c-9fc0-157251872972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:19.115899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065d274-621c-445c-9fc0-157251872972 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:19.119373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065d274-621c-445c-9fc0-157251872972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:19.119702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065d274-621c-445c-9fc0-157251872972 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:19.120389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065d274-621c-445c-9fc0-157251872972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:19.123866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065d274-621c-445c-9fc0-157251872972 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:19.855502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 895/1780] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:18 2022] POST /identity/v3/users => generated 293 bytes in 980 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:19.859422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840aa78a-51ec-4843-853a-014c487fd2d5 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:19.870291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840aa78a-51ec-4843-853a-014c487fd2d5 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:19.876693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086a655-5339-4dfe-8563-7267d0d97ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:19.876958 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086a655-5339-4dfe-8563-7267d0d97ebb admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:19.877112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086a655-5339-4dfe-8563-7267d0d97ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:19.877451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086a655-5339-4dfe-8563-7267d0d97ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:19.881303 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086a655-5339-4dfe-8563-7267d0d97ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:19.885040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086a655-5339-4dfe-8563-7267d0d97ebb admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:19.900039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 886/1781] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:19.902662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065d274-621c-445c-9fc0-157251872972 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:19.920659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065d274-621c-445c-9fc0-157251872972 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:19.938513 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e6552-1424-4dfb-a9c2-6dd0205c6ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:19.939182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e6552-1424-4dfb-a9c2-6dd0205c6ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:19.940566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e6552-1424-4dfb-a9c2-6dd0205c6ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:19.941444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e6552-1424-4dfb-a9c2-6dd0205c6ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:19.941564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e6552-1424-4dfb-a9c2-6dd0205c6ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:19.943421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e6552-1424-4dfb-a9c2-6dd0205c6ae3 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:19.968314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 896/1782] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:19.972548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086a655-5339-4dfe-8563-7267d0d97ebb admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:19.996243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086a655-5339-4dfe-8563-7267d0d97ebb admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:20.019531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f87fb-7a39-4e0d-931d-925267602dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:20.019531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f87fb-7a39-4e0d-931d-925267602dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.019531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f87fb-7a39-4e0d-931d-925267602dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.019531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f87fb-7a39-4e0d-931d-925267602dc5 admin admin] PATH_INFO: `/v3/projects/4ead214ac2af4ac28b4d81d3ad0a61f4/users/af97b440e72545ce9b654148966ea4a2/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.068096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f87fb-7a39-4e0d-931d-925267602dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ead214ac2af4ac28b4d81d3ad0a61f4, user_id=af97b440e72545ce9b654148966ea4a2, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:20.072264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f87fb-7a39-4e0d-931d-925267602dc5 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:20.087181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:20.193217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 887/1783] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:19 2022] PUT /identity/v3/projects/4ead214ac2af4ac28b4d81d3ad0a61f4/users/af97b440e72545ce9b654148966ea4a2/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:20.193401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e6552-1424-4dfb-a9c2-6dd0205c6ae3 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.195228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c33e6552-1424-4dfb-a9c2-6dd0205c6ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:20.209393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8bf9a-63ad-4752-87fc-3f95605f915c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.209802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8bf9a-63ad-4752-87fc-3f95605f915c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.210752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8bf9a-63ad-4752-87fc-3f95605f915c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.210921 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e6552-1424-4dfb-a9c2-6dd0205c6ae3 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:20.254549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-47a8bf9a-63ad-4752-87fc-3f95605f915c None None] MFA Rules not processed for user `af97b440e72545ce9b654148966ea4a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:20.255934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-47a8bf9a-63ad-4752-87fc-3f95605f915c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:20.357021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07232b4-edd6-42ff-a7e6-09a39dc94f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:20.357688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07232b4-edd6-42ff-a7e6-09a39dc94f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.358149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07232b4-edd6-42ff-a7e6-09a39dc94f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.358435 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07232b4-edd6-42ff-a7e6-09a39dc94f5c admin admin] PATH_INFO: `/v3/users/50d4e8a819084459bfc17293e4907880` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.367553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07232b4-edd6-42ff-a7e6-09a39dc94f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d4e8a819084459bfc17293e4907880)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:20.368775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07232b4-edd6-42ff-a7e6-09a39dc94f5c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:20.448724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 888/1784] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:20.450412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8bf9a-63ad-4752-87fc-3f95605f915c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.470402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8bf9a-63ad-4752-87fc-3f95605f915c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:20.476036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024d125-2dc7-4ce2-9a4d-50fcccab6568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:20.477107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024d125-2dc7-4ce2-9a4d-50fcccab6568 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.477604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024d125-2dc7-4ce2-9a4d-50fcccab6568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.478040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024d125-2dc7-4ce2-9a4d-50fcccab6568 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.478830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024d125-2dc7-4ce2-9a4d-50fcccab6568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:20.480954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024d125-2dc7-4ce2-9a4d-50fcccab6568 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:20.646853 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 889/1785] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:20 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:20.648348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc718909-3315-4ee0-ace3-b7eea9e7f11f None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.648739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc718909-3315-4ee0-ace3-b7eea9e7f11f None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.649390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc718909-3315-4ee0-ace3-b7eea9e7f11f None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.651218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 890/1786] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:17:20.651963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc718909-3315-4ee0-ace3-b7eea9e7f11f None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.654585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f07232b4-edd6-42ff-a7e6-09a39dc94f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d4e8a819084459bfc17293e4907880'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:20.655507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f07232b4-edd6-42ff-a7e6-09a39dc94f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d4e8a819084459bfc17293e4907880'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:20.665770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f07232b4-edd6-42ff-a7e6-09a39dc94f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d4e8a819084459bfc17293e4907880'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:20.680258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f07232b4-edd6-42ff-a7e6-09a39dc94f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d4e8a819084459bfc17293e4907880'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:20.724821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc718909-3315-4ee0-ace3-b7eea9e7f11f None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:20.730861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d641b3f-5499-4756-8e95-7b31384111d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:20.731471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d641b3f-5499-4756-8e95-7b31384111d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.731746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d641b3f-5499-4756-8e95-7b31384111d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.732015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d641b3f-5499-4756-8e95-7b31384111d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.732631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d641b3f-5499-4756-8e95-7b31384111d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:20.734603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d641b3f-5499-4756-8e95-7b31384111d2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:20.773306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f07232b4-edd6-42ff-a7e6-09a39dc94f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d4e8a819084459bfc17293e4907880'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:20.781573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 897/1787] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:20 2022] DELETE /identity/v3/users/50d4e8a819084459bfc17293e4907880 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:20.783123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef18702-7be1-44f8-9c7b-6c8af30c0670 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.784568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef18702-7be1-44f8-9c7b-6c8af30c0670 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.784902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef18702-7be1-44f8-9c7b-6c8af30c0670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.815525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3ef18702-7be1-44f8-9c7b-6c8af30c0670 None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:20.817798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3ef18702-7be1-44f8-9c7b-6c8af30c0670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:20.853182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 891/1788] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:20 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:20.856310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1fca-e2f3-4aab-a6e6-b095910221ec None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.857541 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1fca-e2f3-4aab-a6e6-b095910221ec None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.859475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1fca-e2f3-4aab-a6e6-b095910221ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.917053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-4a2e1fca-e2f3-4aab-a6e6-b095910221ec None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:20.918818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-4a2e1fca-e2f3-4aab-a6e6-b095910221ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:20.976174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 898/1789] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:20.977265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef18702-7be1-44f8-9c7b-6c8af30c0670 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.979879 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3ef18702-7be1-44f8-9c7b-6c8af30c0670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:20.998210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 892/1790] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:20.999462 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e1fca-e2f3-4aab-a6e6-b095910221ec None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:21.000555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-4a2e1fca-e2f3-4aab-a6e6-b095910221ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:21.025518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e1fca-e2f3-4aab-a6e6-b095910221ec None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:21.031657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962db054-f521-4920-a7b6-446946ee74c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:21.031908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962db054-f521-4920-a7b6-446946ee74c1 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:21.032184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962db054-f521-4920-a7b6-446946ee74c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:21.032456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962db054-f521-4920-a7b6-446946ee74c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:21.043146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962db054-f521-4920-a7b6-446946ee74c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:21.044965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962db054-f521-4920-a7b6-446946ee74c1 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:21.058238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 893/1791] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:21.059244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962db054-f521-4920-a7b6-446946ee74c1 service nova] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:21.060503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef18702-7be1-44f8-9c7b-6c8af30c0670 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:21.060826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-962db054-f521-4920-a7b6-446946ee74c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:21.066226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35cc28-d749-4ba2-a8e9-aaf6b3247df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:21.067235 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35cc28-d749-4ba2-a8e9-aaf6b3247df9 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:21.073506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35cc28-d749-4ba2-a8e9-aaf6b3247df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:21.073506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35cc28-d749-4ba2-a8e9-aaf6b3247df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:21.073506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35cc28-d749-4ba2-a8e9-aaf6b3247df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:21.073506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35cc28-d749-4ba2-a8e9-aaf6b3247df9 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:21.086668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 899/1792] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:21.091545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35cc28-d749-4ba2-a8e9-aaf6b3247df9 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:21.106196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35cc28-d749-4ba2-a8e9-aaf6b3247df9 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:21.111289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27068f52-83e2-469e-bc63-b5b4a0a63a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:21.112040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27068f52-83e2-469e-bc63-b5b4a0a63a5e admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:21.113123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27068f52-83e2-469e-bc63-b5b4a0a63a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:21.113123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27068f52-83e2-469e-bc63-b5b4a0a63a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:21.113681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27068f52-83e2-469e-bc63-b5b4a0a63a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:21.115766 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27068f52-83e2-469e-bc63-b5b4a0a63a5e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:21.146762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 900/1793] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:21.153326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27068f52-83e2-469e-bc63-b5b4a0a63a5e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:21.170353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962db054-f521-4920-a7b6-446946ee74c1 service nova] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:21.172448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27068f52-83e2-469e-bc63-b5b4a0a63a5e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:21.175075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c80fab-f7c2-4e06-afe2-092d420599b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:21.175715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c80fab-f7c2-4e06-afe2-092d420599b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:21.176130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c80fab-f7c2-4e06-afe2-092d420599b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:21.176411 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c80fab-f7c2-4e06-afe2-092d420599b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:21.181950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e74f9-2109-485b-8cd2-aed5739aa5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:21.183868 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e74f9-2109-485b-8cd2-aed5739aa5e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:21.184494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e74f9-2109-485b-8cd2-aed5739aa5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:21.184923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e74f9-2109-485b-8cd2-aed5739aa5e4 admin admin] PATH_INFO: `/v3/projects/9662704043f641f4bed814c8e438ac73/users/31704c3041a145408693f663ebb7c616/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:21.202387 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c80fab-f7c2-4e06-afe2-092d420599b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:21.208039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c80fab-f7c2-4e06-afe2-092d420599b3 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:21.221955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e74f9-2109-485b-8cd2-aed5739aa5e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9662704043f641f4bed814c8e438ac73, user_id=31704c3041a145408693f663ebb7c616, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:21.225955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e74f9-2109-485b-8cd2-aed5739aa5e4 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:21.248697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 894/1794] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:21.317448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 901/1795] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:21 2022] PUT /identity/v3/projects/9662704043f641f4bed814c8e438ac73/users/31704c3041a145408693f663ebb7c616/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:21.331083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9fef-a5f1-4e7d-981a-23f0f06c45e3 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:21.331830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9fef-a5f1-4e7d-981a-23f0f06c45e3 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:21.332308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9fef-a5f1-4e7d-981a-23f0f06c45e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:21.397925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c90d9fef-a5f1-4e7d-981a-23f0f06c45e3 None None] MFA Rules not processed for user `31704c3041a145408693f663ebb7c616`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:21.399593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c90d9fef-a5f1-4e7d-981a-23f0f06c45e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:21.500016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 895/1796] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:21.504663 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e74f9-2109-485b-8cd2-aed5739aa5e4 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:21.602183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e74f9-2109-485b-8cd2-aed5739aa5e4 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:21.612575 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac130a-087a-4149-bf63-b282578f45bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:21.613800 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac130a-087a-4149-bf63-b282578f45bd admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:21.614180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac130a-087a-4149-bf63-b282578f45bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:21.614510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac130a-087a-4149-bf63-b282578f45bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:21.615272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac130a-087a-4149-bf63-b282578f45bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:21.617465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac130a-087a-4149-bf63-b282578f45bd admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:21.750237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 902/1797] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:21 2022] POST /identity/v3/projects => generated 387 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:21.754681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d9fef-a5f1-4e7d-981a-23f0f06c45e3 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:21.766365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d9fef-a5f1-4e7d-981a-23f0f06c45e3 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:21.789119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac130a-087a-4149-bf63-b282578f45bd admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:21.792037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2dac130a-087a-4149-bf63-b282578f45bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:21.850180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291c4d9-0686-4a55-982c-5a40b1a62349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:21.851043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291c4d9-0686-4a55-982c-5a40b1a62349 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:21.851383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291c4d9-0686-4a55-982c-5a40b1a62349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:21.851664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291c4d9-0686-4a55-982c-5a40b1a62349 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:21.852825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b291c4d9-0686-4a55-982c-5a40b1a62349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:21.854980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b291c4d9-0686-4a55-982c-5a40b1a62349 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:21.872479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:21.896212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac130a-087a-4149-bf63-b282578f45bd admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:21.902307 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:21.902955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:21.903267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:21.903543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] PATH_INFO: `/v3/projects/91172553622a4829948be99ea0295882` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:21.904730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=91172553622a4829948be99ea0295882)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:21.906768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:22.018036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 896/1798] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:21 2022] POST /identity/v3/users => generated 300 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:22.025444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291c4d9-0686-4a55-982c-5a40b1a62349 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:22.029981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291c4d9-0686-4a55-982c-5a40b1a62349 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:22.034966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ee736-5187-471a-9f91-4c9de81d28a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:22.035663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ee736-5187-471a-9f91-4c9de81d28a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:22.036010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ee736-5187-471a-9f91-4c9de81d28a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:22.036348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ee736-5187-471a-9f91-4c9de81d28a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:22.037126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ee736-5187-471a-9f91-4c9de81d28a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:22.038861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ee736-5187-471a-9f91-4c9de81d28a7 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:22.053455 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 897/1799] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:22.059343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ee736-5187-471a-9f91-4c9de81d28a7 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:22.072560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ee736-5187-471a-9f91-4c9de81d28a7 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:22.083520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91172553622a4829948be99ea0295882', 'request_id': 'req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:22.097398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91172553622a4829948be99ea0295882', 'request_id': 'req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:22.103773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:22.187167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91172553622a4829948be99ea0295882', 'request_id': 'req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:22.192491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae0d6d-0dbc-4c3b-9db4-6ff75cf8f5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:22.193486 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae0d6d-0dbc-4c3b-9db4-6ff75cf8f5e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:22.193976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae0d6d-0dbc-4c3b-9db4-6ff75cf8f5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:22.194437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae0d6d-0dbc-4c3b-9db4-6ff75cf8f5e4 admin admin] PATH_INFO: `/v3/projects/0070db64a75f4bf6b421779f0bbd0a83/users/712ca787a4784e6b8df64cb2004ac964/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:22.204006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] The token cache is being invalidate because project 91172553622a4829948be99ea0295882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:22.204398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:22.205220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:22.207122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 903/1800] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:21 2022] DELETE /identity/v3/projects/91172553622a4829948be99ea0295882 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:22.211524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:22.220382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae0d6d-0dbc-4c3b-9db4-6ff75cf8f5e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0070db64a75f4bf6b421779f0bbd0a83, user_id=712ca787a4784e6b8df64cb2004ac964, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:22.220572 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:22.225217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae0d6d-0dbc-4c3b-9db4-6ff75cf8f5e4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:22.241396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf925bf1-f2fd-4528-ba77-4e787c70c5fb admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:22.246813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:22.247463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:22.247753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:22.253049 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] PATH_INFO: `/v3/users/b37e1dc507534091a058031cee67c8d7` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:22.255918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37e1dc507534091a058031cee67c8d7)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:22.259070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:22.327040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 898/1801] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:22 2022] PUT /identity/v3/projects/0070db64a75f4bf6b421779f0bbd0a83/users/712ca787a4784e6b8df64cb2004ac964/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:22.332476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae0d6d-0dbc-4c3b-9db4-6ff75cf8f5e4 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:22.334215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-08ae0d6d-0dbc-4c3b-9db4-6ff75cf8f5e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:22.471252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae0d6d-0dbc-4c3b-9db4-6ff75cf8f5e4 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:22.477729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644fea1-a54d-433b-b39c-7cd4cc0a0af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:22.484142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644fea1-a54d-433b-b39c-7cd4cc0a0af7 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:22.484142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644fea1-a54d-433b-b39c-7cd4cc0a0af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:22.484142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644fea1-a54d-433b-b39c-7cd4cc0a0af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:22.484142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644fea1-a54d-433b-b39c-7cd4cc0a0af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:22.484142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644fea1-a54d-433b-b39c-7cd4cc0a0af7 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:22.507140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 899/1802] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:22.513468 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644fea1-a54d-433b-b39c-7cd4cc0a0af7 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:22.525064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644fea1-a54d-433b-b39c-7cd4cc0a0af7 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:22.530820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729db51-5317-479d-b312-c6d90d63ba98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:22.531510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729db51-5317-479d-b312-c6d90d63ba98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:22.531803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729db51-5317-479d-b312-c6d90d63ba98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:22.532063 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729db51-5317-479d-b312-c6d90d63ba98 admin admin] PATH_INFO: `/v3/projects/0070db64a75f4bf6b421779f0bbd0a83/users/712ca787a4784e6b8df64cb2004ac964/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:22.534252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7729db51-5317-479d-b312-c6d90d63ba98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0070db64a75f4bf6b421779f0bbd0a83, user_id=712ca787a4784e6b8df64cb2004ac964, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:22.537098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7729db51-5317-479d-b312-c6d90d63ba98 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:22.544984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37e1dc507534091a058031cee67c8d7'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:22.551049 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37e1dc507534091a058031cee67c8d7'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:22.558026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 900/1803] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:22 2022] PUT /identity/v3/projects/0070db64a75f4bf6b421779f0bbd0a83/users/712ca787a4784e6b8df64cb2004ac964/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:22.570024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f48be-902a-46c3-9705-e9085c36a9fc None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:22.570362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f48be-902a-46c3-9705-e9085c36a9fc None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:22.570645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f48be-902a-46c3-9705-e9085c36a9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:22.580500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37e1dc507534091a058031cee67c8d7'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:22.610335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-ec6f48be-902a-46c3-9705-e9085c36a9fc None None] MFA Rules not processed for user `712ca787a4784e6b8df64cb2004ac964`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:22.612051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ec6f48be-902a-46c3-9705-e9085c36a9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:22.613206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37e1dc507534091a058031cee67c8d7'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:22.700743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37e1dc507534091a058031cee67c8d7'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:22.704478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 901/1804] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:22.707184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 904/1805] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:22 2022] DELETE /identity/v3/users/b37e1dc507534091a058031cee67c8d7 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:22.716655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f48be-902a-46c3-9705-e9085c36a9fc None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:22.718161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ec6f48be-902a-46c3-9705-e9085c36a9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:22.809023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f48be-902a-46c3-9705-e9085c36a9fc None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:22.891670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a57c12-fbcb-4396-aa62-bcb4d3826e32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:22.892480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a57c12-fbcb-4396-aa62-bcb4d3826e32 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:22.892830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a57c12-fbcb-4396-aa62-bcb4d3826e32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:22.893237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a57c12-fbcb-4396-aa62-bcb4d3826e32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:22.898542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-70a57c12-fbcb-4396-aa62-bcb4d3826e32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:22.997442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a57c12-fbcb-4396-aa62-bcb4d3826e32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:22.998952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a57c12-fbcb-4396-aa62-bcb4d3826e32 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:23.029329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 902/1806] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:23.304636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:23.346141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:23.347597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:23.483374 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cee2c1-6924-458e-826d-99b679239a2e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:23.497813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:23.498479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:23.498933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:23.499239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] PATH_INFO: `/v3/projects/a287ced976894d63bb9c31d833362bec` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:23.501109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a287ced976894d63bb9c31d833362bec)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:23.503730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:23.665418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:23.701778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a287ced976894d63bb9c31d833362bec', 'request_id': 'req-0757917f-1b39-4641-9495-9d5e1c44f50e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:23.718117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a287ced976894d63bb9c31d833362bec', 'request_id': 'req-0757917f-1b39-4641-9495-9d5e1c44f50e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:23.849606 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a287ced976894d63bb9c31d833362bec', 'request_id': 'req-0757917f-1b39-4641-9495-9d5e1c44f50e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:23.865344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] The token cache is being invalidate because project a287ced976894d63bb9c31d833362bec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:23.866071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:23.866900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0757917f-1b39-4641-9495-9d5e1c44f50e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:23.870063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 905/1807] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:23 2022] DELETE /identity/v3/projects/a287ced976894d63bb9c31d833362bec => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:23.875648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c24840-d4f5-480a-843e-551f21eaed56 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:23.875964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c24840-d4f5-480a-843e-551f21eaed56 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:23.876243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c24840-d4f5-480a-843e-551f21eaed56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:23.900467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-99c24840-d4f5-480a-843e-551f21eaed56 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:23.901797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-99c24840-d4f5-480a-843e-551f21eaed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:23.989995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 903/1808] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:24.000059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900e738-1e8b-481a-9056-f9efd8df2b54 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:24.000356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900e738-1e8b-481a-9056-f9efd8df2b54 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:24.000666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900e738-1e8b-481a-9056-f9efd8df2b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:24.030412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3900e738-1e8b-481a-9056-f9efd8df2b54 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:24.031369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3900e738-1e8b-481a-9056-f9efd8df2b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:24.039729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 906/1809] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:24.044094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c24840-d4f5-480a-843e-551f21eaed56 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:24.057347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c24840-d4f5-480a-843e-551f21eaed56 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:24.069150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd982a1-b25a-4f20-b003-210b38a145a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:24.069810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd982a1-b25a-4f20-b003-210b38a145a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:24.070084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd982a1-b25a-4f20-b003-210b38a145a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:24.070445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd982a1-b25a-4f20-b003-210b38a145a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:24.071832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd982a1-b25a-4f20-b003-210b38a145a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:24.072134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd982a1-b25a-4f20-b003-210b38a145a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:24.073773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd982a1-b25a-4f20-b003-210b38a145a2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:24.090141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 904/1810] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:24.095770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900e738-1e8b-481a-9056-f9efd8df2b54 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:24.109762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900e738-1e8b-481a-9056-f9efd8df2b54 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:24.115139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63272f-5b90-4cb6-82b3-3cac7e51e829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:24.116118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63272f-5b90-4cb6-82b3-3cac7e51e829 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:24.116494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63272f-5b90-4cb6-82b3-3cac7e51e829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:24.116767 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63272f-5b90-4cb6-82b3-3cac7e51e829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:24.117429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63272f-5b90-4cb6-82b3-3cac7e51e829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:24.120710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63272f-5b90-4cb6-82b3-3cac7e51e829 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:24.233964 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 907/1811] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:24 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:24.238138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd982a1-b25a-4f20-b003-210b38a145a2 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:24.260367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd982a1-b25a-4f20-b003-210b38a145a2 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:24.389686 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed51659-d623-4290-b51f-5eea6f747755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:24.390331 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed51659-d623-4290-b51f-5eea6f747755 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:24.390656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed51659-d623-4290-b51f-5eea6f747755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:24.391249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed51659-d623-4290-b51f-5eea6f747755 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:24.391926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed51659-d623-4290-b51f-5eea6f747755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:24.394031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed51659-d623-4290-b51f-5eea6f747755 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:24.524802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 905/1812] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:24 2022] POST /identity/v3/users => generated 290 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:24.529562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63272f-5b90-4cb6-82b3-3cac7e51e829 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:24.544639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63272f-5b90-4cb6-82b3-3cac7e51e829 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:24.572301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f19386-a3d1-4657-ab30-3b0f345a87aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:24.572852 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f19386-a3d1-4657-ab30-3b0f345a87aa admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:24.573159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f19386-a3d1-4657-ab30-3b0f345a87aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:24.573436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f19386-a3d1-4657-ab30-3b0f345a87aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:24.574101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f19386-a3d1-4657-ab30-3b0f345a87aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:24.576106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f19386-a3d1-4657-ab30-3b0f345a87aa admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:24.618880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 908/1813] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:24.623006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed51659-d623-4290-b51f-5eea6f747755 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:24.639745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed51659-d623-4290-b51f-5eea6f747755 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:24.669384 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7272d-7141-4bce-8518-1d1bc82b1306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:24.670008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7272d-7141-4bce-8518-1d1bc82b1306 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:24.670315 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7272d-7141-4bce-8518-1d1bc82b1306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:24.670596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7272d-7141-4bce-8518-1d1bc82b1306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:24.671931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7272d-7141-4bce-8518-1d1bc82b1306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:24.692303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f19386-a3d1-4657-ab30-3b0f345a87aa admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:24.694400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-68f19386-a3d1-4657-ab30-3b0f345a87aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:24.699646 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7272d-7141-4bce-8518-1d1bc82b1306 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:24.714214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:24.724253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 906/1814] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:24.725218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7272d-7141-4bce-8518-1d1bc82b1306 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:24.756124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7272d-7141-4bce-8518-1d1bc82b1306 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:24.784082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456660-192e-4efb-8bb1-d57ccdc3b47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:24.784784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456660-192e-4efb-8bb1-d57ccdc3b47b admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:24.785103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456660-192e-4efb-8bb1-d57ccdc3b47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:24.785383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456660-192e-4efb-8bb1-d57ccdc3b47b admin admin] PATH_INFO: `/v3/projects/496cf4043b844c76a9c2d389b74cf172/users/7caa5b45f57e443a8d3bfe24cfd145d3/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:24.807474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f19386-a3d1-4657-ab30-3b0f345a87aa admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:24.810252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd456660-192e-4efb-8bb1-d57ccdc3b47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=496cf4043b844c76a9c2d389b74cf172, user_id=7caa5b45f57e443a8d3bfe24cfd145d3, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:24.811931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4bbb7-d615-4a09-bdd3-f4df271a8560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:24.812733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4bbb7-d615-4a09-bdd3-f4df271a8560 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:24.813195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4bbb7-d615-4a09-bdd3-f4df271a8560 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:24.813442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd456660-192e-4efb-8bb1-d57ccdc3b47b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:24.813933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4bbb7-d615-4a09-bdd3-f4df271a8560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:24.822454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-86c4bbb7-d615-4a09-bdd3-f4df271a8560 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:24.906187 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4bbb7-d615-4a09-bdd3-f4df271a8560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:24.907689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4bbb7-d615-4a09-bdd3-f4df271a8560 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:24.917516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 909/1815] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:24.983223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 907/1816] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:24 2022] PUT /identity/v3/projects/496cf4043b844c76a9c2d389b74cf172/users/7caa5b45f57e443a8d3bfe24cfd145d3/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:24.994329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406aa76-989c-431d-a608-8ed63dfbd4b7 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:24.994615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406aa76-989c-431d-a608-8ed63dfbd4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:24.994914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406aa76-989c-431d-a608-8ed63dfbd4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:25.045117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e406aa76-989c-431d-a608-8ed63dfbd4b7 None None] MFA Rules not processed for user `7caa5b45f57e443a8d3bfe24cfd145d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:25.049724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e406aa76-989c-431d-a608-8ed63dfbd4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:25.127156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 910/1817] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:25.131852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456660-192e-4efb-8bb1-d57ccdc3b47b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:25.199740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456660-192e-4efb-8bb1-d57ccdc3b47b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:25.213236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b5f22-00ab-4836-9ada-77dc2a593768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:25.214227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b5f22-00ab-4836-9ada-77dc2a593768 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:25.215037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b5f22-00ab-4836-9ada-77dc2a593768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:25.215487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b5f22-00ab-4836-9ada-77dc2a593768 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:25.216311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b5f22-00ab-4836-9ada-77dc2a593768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:25.218720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b5f22-00ab-4836-9ada-77dc2a593768 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:25.326923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 908/1818] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:25 2022] POST /identity/v3/projects => generated 379 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:25.331057 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406aa76-989c-431d-a608-8ed63dfbd4b7 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:25.343352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406aa76-989c-431d-a608-8ed63dfbd4b7 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:25.416832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:25.466374 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96ef9e-7c13-4ab1-96f4-4013d492f080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:25.467119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96ef9e-7c13-4ab1-96f4-4013d492f080 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:25.467464 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96ef9e-7c13-4ab1-96f4-4013d492f080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:25.469258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96ef9e-7c13-4ab1-96f4-4013d492f080 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:25.470074 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96ef9e-7c13-4ab1-96f4-4013d492f080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:25.474802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96ef9e-7c13-4ab1-96f4-4013d492f080 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:25.573921 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 911/1819] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:25 2022] POST /identity/v3/users => generated 296 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:25.578643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b5f22-00ab-4836-9ada-77dc2a593768 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:25.589304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b5f22-00ab-4836-9ada-77dc2a593768 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:25.595227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accc480-c064-48cc-b227-eac5c221bf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:25.596949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accc480-c064-48cc-b227-eac5c221bf5c admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:25.597139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accc480-c064-48cc-b227-eac5c221bf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:25.597584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accc480-c064-48cc-b227-eac5c221bf5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:25.599799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accc480-c064-48cc-b227-eac5c221bf5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:25.604793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accc480-c064-48cc-b227-eac5c221bf5c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:25.628534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 909/1820] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:25.632974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96ef9e-7c13-4ab1-96f4-4013d492f080 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:25.649605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96ef9e-7c13-4ab1-96f4-4013d492f080 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:25.656748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b45b2f-e3ba-4908-bf91-965cc1671693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:25.666607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b45b2f-e3ba-4908-bf91-965cc1671693 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:25.666607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b45b2f-e3ba-4908-bf91-965cc1671693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:25.666607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b45b2f-e3ba-4908-bf91-965cc1671693 admin admin] PATH_INFO: `/v3/projects/7a193770d410416a944a802a1407304e/users/0e4555deb6ce44b382c12061be7c7de3/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:25.687683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b45b2f-e3ba-4908-bf91-965cc1671693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a193770d410416a944a802a1407304e, user_id=0e4555deb6ce44b382c12061be7c7de3, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:25.697194 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b45b2f-e3ba-4908-bf91-965cc1671693 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:25.824711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 912/1821] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:25 2022] PUT /identity/v3/projects/7a193770d410416a944a802a1407304e/users/0e4555deb6ce44b382c12061be7c7de3/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:25.828944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accc480-c064-48cc-b227-eac5c221bf5c admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:25.842474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accc480-c064-48cc-b227-eac5c221bf5c admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:25.981529 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701ed12-15eb-4dcb-9785-6a65af590620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:25.982151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701ed12-15eb-4dcb-9785-6a65af590620 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:25.982464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701ed12-15eb-4dcb-9785-6a65af590620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:25.982745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701ed12-15eb-4dcb-9785-6a65af590620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:25.983389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701ed12-15eb-4dcb-9785-6a65af590620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:25.985188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701ed12-15eb-4dcb-9785-6a65af590620 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:25.997881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 910/1822] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:26.002640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b45b2f-e3ba-4908-bf91-965cc1671693 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:26.015559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b45b2f-e3ba-4908-bf91-965cc1671693 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:26.022031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612201e-313f-4ffc-b55b-3d9b8bfc055d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:26.022898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612201e-313f-4ffc-b55b-3d9b8bfc055d admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:26.023346 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612201e-313f-4ffc-b55b-3d9b8bfc055d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:26.023691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612201e-313f-4ffc-b55b-3d9b8bfc055d admin admin] PATH_INFO: `/v3/projects/7a193770d410416a944a802a1407304e/users/0e4555deb6ce44b382c12061be7c7de3/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:26.027117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612201e-313f-4ffc-b55b-3d9b8bfc055d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a193770d410416a944a802a1407304e, user_id=0e4555deb6ce44b382c12061be7c7de3, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:26.029926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612201e-313f-4ffc-b55b-3d9b8bfc055d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:26.050541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 913/1823] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:26 2022] PUT /identity/v3/projects/7a193770d410416a944a802a1407304e/users/0e4555deb6ce44b382c12061be7c7de3/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:26.061667 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077bd2f-4cfa-48ad-ae67-24ae92332b3b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:26.061965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077bd2f-4cfa-48ad-ae67-24ae92332b3b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:26.062252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077bd2f-4cfa-48ad-ae67-24ae92332b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:26.098401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-3077bd2f-4cfa-48ad-ae67-24ae92332b3b None None] MFA Rules not processed for user `0e4555deb6ce44b382c12061be7c7de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:26.099837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3077bd2f-4cfa-48ad-ae67-24ae92332b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:26.194035 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 911/1824] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:26.462561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:27.165323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:27.239412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:28.412881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:29.953542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:29.954840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612201e-313f-4ffc-b55b-3d9b8bfc055d admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:29.956530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8612201e-313f-4ffc-b55b-3d9b8bfc055d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:30.026270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612201e-313f-4ffc-b55b-3d9b8bfc055d admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:30.030812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a055517-69e1-4c66-b9c4-e91febde1164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:30.031433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a055517-69e1-4c66-b9c4-e91febde1164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:30.031713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a055517-69e1-4c66-b9c4-e91febde1164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:30.031987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a055517-69e1-4c66-b9c4-e91febde1164 admin admin] PATH_INFO: `/v3/users/831114b04211479b8a73677d6ed018e9` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:30.033126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a055517-69e1-4c66-b9c4-e91febde1164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=831114b04211479b8a73677d6ed018e9)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:30.035047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a055517-69e1-4c66-b9c4-e91febde1164 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:30.217609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077bd2f-4cfa-48ad-ae67-24ae92332b3b None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:30.219178 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3077bd2f-4cfa-48ad-ae67-24ae92332b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:30.235580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077bd2f-4cfa-48ad-ae67-24ae92332b3b None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:30.240597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:30.241563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:30.243155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:30.243613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] PATH_INFO: `/v3/users/4851c4ac15784cba81d6b74166880e99` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:30.245131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4851c4ac15784cba81d6b74166880e99)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:30.247842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:30.288989 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1a055517-69e1-4c66-b9c4-e91febde1164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831114b04211479b8a73677d6ed018e9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:30.291422 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1a055517-69e1-4c66-b9c4-e91febde1164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831114b04211479b8a73677d6ed018e9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:30.301567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1a055517-69e1-4c66-b9c4-e91febde1164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831114b04211479b8a73677d6ed018e9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:30.331436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1a055517-69e1-4c66-b9c4-e91febde1164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831114b04211479b8a73677d6ed018e9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:30.366856 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:30.476101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1a055517-69e1-4c66-b9c4-e91febde1164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831114b04211479b8a73677d6ed018e9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:30.499385 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 914/1825] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:29 2022] DELETE /identity/v3/users/831114b04211479b8a73677d6ed018e9 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:30.503966 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524576db-d991-4cf3-9785-da3593412a28 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:30.504255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524576db-d991-4cf3-9785-da3593412a28 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:30.504532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524576db-d991-4cf3-9785-da3593412a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:30.525683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-524576db-d991-4cf3-9785-da3593412a28 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:30.526753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-524576db-d991-4cf3-9785-da3593412a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:30.536025 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4851c4ac15784cba81d6b74166880e99'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:30.537049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4851c4ac15784cba81d6b74166880e99'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:30.551092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4851c4ac15784cba81d6b74166880e99'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:30.551092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4851c4ac15784cba81d6b74166880e99'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:30.604899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 915/1826] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:30.625764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524576db-d991-4cf3-9785-da3593412a28 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:30.627370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-524576db-d991-4cf3-9785-da3593412a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:30.700134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524576db-d991-4cf3-9785-da3593412a28 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:30.704618 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6255ff-082b-48cf-b528-bbe218964702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:30.705310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6255ff-082b-48cf-b528-bbe218964702 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:30.705992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6255ff-082b-48cf-b528-bbe218964702 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:30.705992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6255ff-082b-48cf-b528-bbe218964702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:30.709734 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4851c4ac15784cba81d6b74166880e99'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:30.729978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6255ff-082b-48cf-b528-bbe218964702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:30.732390 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6255ff-082b-48cf-b528-bbe218964702 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:30.747166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 916/1827] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:30.766205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 912/1828] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:30 2022] DELETE /identity/v3/users/4851c4ac15784cba81d6b74166880e99 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:30.770742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c5a8a-3ea6-4600-8d35-017434ba0590 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:30.771037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c5a8a-3ea6-4600-8d35-017434ba0590 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:30.771973 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c5a8a-3ea6-4600-8d35-017434ba0590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:30.813758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-d25c5a8a-3ea6-4600-8d35-017434ba0590 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:30.814933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d25c5a8a-3ea6-4600-8d35-017434ba0590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:30.907290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 917/1829] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:30.939500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:31.016669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e5c4e-8044-4ad2-99ba-ffed9bf2252b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:31.032548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e79247-fe8a-4a3c-841a-07635e35a007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:31.033422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e79247-fe8a-4a3c-841a-07635e35a007 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.033908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e79247-fe8a-4a3c-841a-07635e35a007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.034087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e79247-fe8a-4a3c-841a-07635e35a007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.047132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e79247-fe8a-4a3c-841a-07635e35a007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:31.048728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e79247-fe8a-4a3c-841a-07635e35a007 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:31.062409 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 913/1830] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:31.556456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c5a8a-3ea6-4600-8d35-017434ba0590 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:31.558291 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d25c5a8a-3ea6-4600-8d35-017434ba0590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:31.621264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c5a8a-3ea6-4600-8d35-017434ba0590 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:31.627204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb5e4-d766-4a5a-bca8-931df505ec5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:31.627964 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb5e4-d766-4a5a-bca8-931df505ec5e service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.628688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb5e4-d766-4a5a-bca8-931df505ec5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.629181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb5e4-d766-4a5a-bca8-931df505ec5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.631282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-72fbb5e4-d766-4a5a-bca8-931df505ec5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:31.735465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbb5e4-d766-4a5a-bca8-931df505ec5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:31.737055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbb5e4-d766-4a5a-bca8-931df505ec5e service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:31.747721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 918/1831] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:32.370668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e79247-fe8a-4a3c-841a-07635e35a007 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:32.372393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-50e79247-fe8a-4a3c-841a-07635e35a007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:32.389930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e79247-fe8a-4a3c-841a-07635e35a007 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:32.395384 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:32.396233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:32.396680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:32.396995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] PATH_INFO: `/v3/projects/5309ab7ca500477685fe9e5187b11db2` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:32.398184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5309ab7ca500477685fe9e5187b11db2)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:32.400726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:32.582889 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5309ab7ca500477685fe9e5187b11db2', 'request_id': 'req-197faba7-59d7-4adf-8141-abecc051d88b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:32.589435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5309ab7ca500477685fe9e5187b11db2', 'request_id': 'req-197faba7-59d7-4adf-8141-abecc051d88b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:32.596411 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5309ab7ca500477685fe9e5187b11db2', 'request_id': 'req-197faba7-59d7-4adf-8141-abecc051d88b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:32.719671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb5e4-d766-4a5a-bca8-931df505ec5e service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:32.721362 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-72fbb5e4-d766-4a5a-bca8-931df505ec5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:32.741281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb5e4-d766-4a5a-bca8-931df505ec5e service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:32.744538 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] The token cache is being invalidate because project 5309ab7ca500477685fe9e5187b11db2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:32.744976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:32.746670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:32.752972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 914/1832] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:32 2022] DELETE /identity/v3/projects/5309ab7ca500477685fe9e5187b11db2 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:32.758576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:32.764098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:32.830119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:32.830614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:32.831062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:32.831608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] PATH_INFO: `/v3/projects/58353a1c5ff744a99a3814223bdddee8` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:32.833156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58353a1c5ff744a99a3814223bdddee8)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:32.837590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:32.912272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197faba7-59d7-4adf-8141-abecc051d88b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:32.919641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:32.920452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:32.920792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:32.921098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] PATH_INFO: `/v3/users/4219b52ccf3b43d795fc94fcc8067b9d` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:32.923372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4219b52ccf3b43d795fc94fcc8067b9d)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:32.926096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:33.032560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58353a1c5ff744a99a3814223bdddee8', 'request_id': 'req-5247455e-e103-48d2-bdbd-02b498777672'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.042735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58353a1c5ff744a99a3814223bdddee8', 'request_id': 'req-5247455e-e103-48d2-bdbd-02b498777672'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.201400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58353a1c5ff744a99a3814223bdddee8', 'request_id': 'req-5247455e-e103-48d2-bdbd-02b498777672'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.243754 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4219b52ccf3b43d795fc94fcc8067b9d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.247139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4219b52ccf3b43d795fc94fcc8067b9d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.248670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] The token cache is being invalidate because project 58353a1c5ff744a99a3814223bdddee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:33.249069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.249844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.252890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 919/1833] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:32 2022] DELETE /identity/v3/projects/58353a1c5ff744a99a3814223bdddee8 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:33.269285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:33.272164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:33.273004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4219b52ccf3b43d795fc94fcc8067b9d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.274173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4219b52ccf3b43d795fc94fcc8067b9d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.358888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247455e-e103-48d2-bdbd-02b498777672 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:33.364239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:33.365025 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:33.365369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:33.367021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] PATH_INFO: `/v3/users/c29f28b1c3da4ddb9cc497333555ad35` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:33.367885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29f28b1c3da4ddb9cc497333555ad35)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:33.370723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:33.398606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4219b52ccf3b43d795fc94fcc8067b9d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.442106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 915/1834] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:32 2022] DELETE /identity/v3/users/4219b52ccf3b43d795fc94fcc8067b9d => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:33.703687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29f28b1c3da4ddb9cc497333555ad35'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.704463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29f28b1c3da4ddb9cc497333555ad35'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.713072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29f28b1c3da4ddb9cc497333555ad35'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.728022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29f28b1c3da4ddb9cc497333555ad35'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.812829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29f28b1c3da4ddb9cc497333555ad35'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:33.819463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 920/1835] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:33 2022] DELETE /identity/v3/users/c29f28b1c3da4ddb9cc497333555ad35 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:34.239536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:34.241113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:34.339153 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5119bdf-1229-453c-af55-463e3497a42f admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:34.419917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:34.420715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:34.421286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:34.421816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] PATH_INFO: `/v3/projects/e43745a174ae4fd6b86af2f1a66a3b7f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:34.423076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43745a174ae4fd6b86af2f1a66a3b7f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:34.425405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:34.701148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43745a174ae4fd6b86af2f1a66a3b7f', 'request_id': 'req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:34.701148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43745a174ae4fd6b86af2f1a66a3b7f', 'request_id': 'req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:34.704787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43745a174ae4fd6b86af2f1a66a3b7f', 'request_id': 'req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:34.944639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:34.946172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:34.960079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e388c6-002e-4b59-8e47-33ca5f604b99 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:35.033595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:35.034514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:35.034969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:35.035405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] PATH_INFO: `/v3/projects/1ecac7aedfe142b880f8c72e2206bb13` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:35.036748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ecac7aedfe142b880f8c72e2206bb13)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:35.038710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.307714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] The token cache is being invalidate because project e43745a174ae4fd6b86af2f1a66a3b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:36.307714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:36.308292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4571e0e-3eb5-4495-aceb-95d3a0c73c74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:36.310653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 916/1836] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:34 2022] DELETE /identity/v3/projects/e43745a174ae4fd6b86af2f1a66a3b7f => generated 0 bytes in 2072 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:36.317146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026de119-fa08-4df5-a770-eab8fabad87c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.317434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026de119-fa08-4df5-a770-eab8fabad87c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.317717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026de119-fa08-4df5-a770-eab8fabad87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.331716 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ecac7aedfe142b880f8c72e2206bb13', 'request_id': 'req-74ee973d-8a0b-4baa-9d8a-de64a93ee763'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:36.346052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ecac7aedfe142b880f8c72e2206bb13', 'request_id': 'req-74ee973d-8a0b-4baa-9d8a-de64a93ee763'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:36.364887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-026de119-fa08-4df5-a770-eab8fabad87c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:36.366130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-026de119-fa08-4df5-a770-eab8fabad87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:36.474505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ecac7aedfe142b880f8c72e2206bb13', 'request_id': 'req-74ee973d-8a0b-4baa-9d8a-de64a93ee763'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:36.482972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 917/1837] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:36.490630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef5319-2a74-4533-bf6f-765415cdd435 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.490918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef5319-2a74-4533-bf6f-765415cdd435 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.491194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef5319-2a74-4533-bf6f-765415cdd435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.497500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] The token cache is being invalidate because project 1ecac7aedfe142b880f8c72e2206bb13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:36.497965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:36.498772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-74ee973d-8a0b-4baa-9d8a-de64a93ee763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:36.509956 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:36.510614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 921/1838] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:34 2022] DELETE /identity/v3/projects/1ecac7aedfe142b880f8c72e2206bb13 => generated 0 bytes in 1557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:36.512231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4699b0-4068-448e-a1a0-2b052e4d7449 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.512614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4699b0-4068-448e-a1a0-2b052e4d7449 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.514425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4699b0-4068-448e-a1a0-2b052e4d7449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.526013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-0aef5319-2a74-4533-bf6f-765415cdd435 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:36.527821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0aef5319-2a74-4533-bf6f-765415cdd435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:36.537351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 918/1839] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:36.541795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef5319-2a74-4533-bf6f-765415cdd435 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.543393 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-aa4699b0-4068-448e-a1a0-2b052e4d7449 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:36.544607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-aa4699b0-4068-448e-a1a0-2b052e4d7449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:36.553198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef5319-2a74-4533-bf6f-765415cdd435 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:36.557797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 922/1840] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:36.565424 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c3eae-8868-4c35-8d1d-4fe7d919479c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.565789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c3eae-8868-4c35-8d1d-4fe7d919479c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.568006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c3eae-8868-4c35-8d1d-4fe7d919479c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.572840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f7f10-5837-453d-b6a7-be3b2676eb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:36.574756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f7f10-5837-453d-b6a7-be3b2676eb2e admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.575141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f7f10-5837-453d-b6a7-be3b2676eb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.576915 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f7f10-5837-453d-b6a7-be3b2676eb2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.578029 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f7f10-5837-453d-b6a7-be3b2676eb2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:36.578526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f7f10-5837-453d-b6a7-be3b2676eb2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:36.581703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f7f10-5837-453d-b6a7-be3b2676eb2e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.619534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 919/1841] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:17:36.626670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f7f10-5837-453d-b6a7-be3b2676eb2e admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.629409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-490c3eae-8868-4c35-8d1d-4fe7d919479c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:36.629409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-490c3eae-8868-4c35-8d1d-4fe7d919479c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:36.644423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f7f10-5837-453d-b6a7-be3b2676eb2e admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:36.652433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019101cd-3e25-4d1d-97ef-8fb549876eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:36.656941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019101cd-3e25-4d1d-97ef-8fb549876eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.656941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019101cd-3e25-4d1d-97ef-8fb549876eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.656941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019101cd-3e25-4d1d-97ef-8fb549876eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.656941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019101cd-3e25-4d1d-97ef-8fb549876eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:36.660381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 923/1842] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:36.660551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019101cd-3e25-4d1d-97ef-8fb549876eaf admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.662456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c3eae-8868-4c35-8d1d-4fe7d919479c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.681516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c3eae-8868-4c35-8d1d-4fe7d919479c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:36.687544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b8626-16c7-4118-8dd9-d5c13665d104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:36.688235 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b8626-16c7-4118-8dd9-d5c13665d104 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.688521 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b8626-16c7-4118-8dd9-d5c13665d104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.688797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b8626-16c7-4118-8dd9-d5c13665d104 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.690159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b8626-16c7-4118-8dd9-d5c13665d104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:36.690748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b8626-16c7-4118-8dd9-d5c13665d104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:36.692927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b8626-16c7-4118-8dd9-d5c13665d104 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.776911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 924/1843] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:36.778042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b8626-16c7-4118-8dd9-d5c13665d104 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.803211 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b8626-16c7-4118-8dd9-d5c13665d104 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:36.817977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfaccb-69c0-4f4a-a41f-bbde9b0658e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:36.818791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfaccb-69c0-4f4a-a41f-bbde9b0658e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.820019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfaccb-69c0-4f4a-a41f-bbde9b0658e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.820019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfaccb-69c0-4f4a-a41f-bbde9b0658e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.820773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfaccb-69c0-4f4a-a41f-bbde9b0658e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:36.822866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfaccb-69c0-4f4a-a41f-bbde9b0658e2 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.847287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:36.924805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 920/1844] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:36 2022] POST /identity/v3/projects => generated 355 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:36.928010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108ad76-ff28-4bb5-b81d-c74fca84744b None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.928010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108ad76-ff28-4bb5-b81d-c74fca84744b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.928010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108ad76-ff28-4bb5-b81d-c74fca84744b None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.928645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 921/1845] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:17:36.929484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108ad76-ff28-4bb5-b81d-c74fca84744b None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.994816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108ad76-ff28-4bb5-b81d-c74fca84744b None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.001939 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112999b-c624-4ec4-83fc-d58929d66cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.002598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112999b-c624-4ec4-83fc-d58929d66cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.002861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112999b-c624-4ec4-83fc-d58929d66cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.003138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112999b-c624-4ec4-83fc-d58929d66cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.004037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112999b-c624-4ec4-83fc-d58929d66cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.006247 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112999b-c624-4ec4-83fc-d58929d66cc8 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.113367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 925/1846] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:36 2022] POST /identity/v3/projects => generated 385 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:37.113367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd74c4b-bbb1-40c1-acde-9d4f0eb82f81 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.113367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd74c4b-bbb1-40c1-acde-9d4f0eb82f81 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.113367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd74c4b-bbb1-40c1-acde-9d4f0eb82f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.137986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ffd74c4b-bbb1-40c1-acde-9d4f0eb82f81 None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:37.144562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ffd74c4b-bbb1-40c1-acde-9d4f0eb82f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:37.144713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 922/1847] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:36 2022] POST /identity/v3/users => generated 278 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:37.144713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112999b-c624-4ec4-83fc-d58929d66cc8 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:37.154481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112999b-c624-4ec4-83fc-d58929d66cc8 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.232579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3551a48-65b4-400a-87c1-36b6d4fba822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.233426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3551a48-65b4-400a-87c1-36b6d4fba822 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.233712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3551a48-65b4-400a-87c1-36b6d4fba822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.233985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3551a48-65b4-400a-87c1-36b6d4fba822 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.234674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3551a48-65b4-400a-87c1-36b6d4fba822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.234797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 926/1848] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:17:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:37.236727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3551a48-65b4-400a-87c1-36b6d4fba822 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.237834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd74c4b-bbb1-40c1-acde-9d4f0eb82f81 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:37.250449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd74c4b-bbb1-40c1-acde-9d4f0eb82f81 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.255062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e077534-c004-48e1-9a01-d2070e326e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.255682 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e077534-c004-48e1-9a01-d2070e326e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.255942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e077534-c004-48e1-9a01-d2070e326e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.256480 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e077534-c004-48e1-9a01-d2070e326e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.257162 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e077534-c004-48e1-9a01-d2070e326e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.259281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e077534-c004-48e1-9a01-d2070e326e3d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.269627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 927/1849] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:37.270583 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e077534-c004-48e1-9a01-d2070e326e3d admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:37.272219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2e077534-c004-48e1-9a01-d2070e326e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:37.288167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e077534-c004-48e1-9a01-d2070e326e3d admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.294540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daf737-4347-471a-8b4e-a2a2a2e46fbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.295146 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daf737-4347-471a-8b4e-a2a2a2e46fbf service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.295432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daf737-4347-471a-8b4e-a2a2a2e46fbf service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.295702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daf737-4347-471a-8b4e-a2a2a2e46fbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.304641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58daf737-4347-471a-8b4e-a2a2a2e46fbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.306582 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58daf737-4347-471a-8b4e-a2a2a2e46fbf service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.316708 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 928/1850] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:37.317522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daf737-4347-471a-8b4e-a2a2a2e46fbf service nova] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:37.328385 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daf737-4347-471a-8b4e-a2a2a2e46fbf service nova] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.333026 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0a7b7-fafe-4305-9bc6-31fce4dfeae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.333550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0a7b7-fafe-4305-9bc6-31fce4dfeae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.333793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0a7b7-fafe-4305-9bc6-31fce4dfeae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.334069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0a7b7-fafe-4305-9bc6-31fce4dfeae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.334809 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0a7b7-fafe-4305-9bc6-31fce4dfeae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.336383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0a7b7-fafe-4305-9bc6-31fce4dfeae4 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.348872 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 929/1851] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:37.353404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0a7b7-fafe-4305-9bc6-31fce4dfeae4 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:37.363323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0a7b7-fafe-4305-9bc6-31fce4dfeae4 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.368367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b133c-5210-4857-b131-960f815db3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.369659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b133c-5210-4857-b131-960f815db3c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.369659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b133c-5210-4857-b131-960f815db3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.369659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b133c-5210-4857-b131-960f815db3c6 admin admin] PATH_INFO: `/v3/projects/419d159f16d645519676003ef0af8e6e/users/f1f654b76e8a4f1081800e0ddd3a552b/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.373198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 923/1852] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:37 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:37.377493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3551a48-65b4-400a-87c1-36b6d4fba822 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:37.386224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b133c-5210-4857-b131-960f815db3c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=419d159f16d645519676003ef0af8e6e, user_id=f1f654b76e8a4f1081800e0ddd3a552b, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.388243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3551a48-65b4-400a-87c1-36b6d4fba822 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.389044 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b133c-5210-4857-b131-960f815db3c6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.393432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79de38-1558-4761-8fe4-2f3a57cb96bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.394105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79de38-1558-4761-8fe4-2f3a57cb96bf admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.394400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79de38-1558-4761-8fe4-2f3a57cb96bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.394666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79de38-1558-4761-8fe4-2f3a57cb96bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.395310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79de38-1558-4761-8fe4-2f3a57cb96bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.396943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79de38-1558-4761-8fe4-2f3a57cb96bf admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.408487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 924/1853] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:37.412611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79de38-1558-4761-8fe4-2f3a57cb96bf admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:37.422554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79de38-1558-4761-8fe4-2f3a57cb96bf admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.427269 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c71c5-bf71-4972-91ee-66f750490c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.427840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c71c5-bf71-4972-91ee-66f750490c1a admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.428146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c71c5-bf71-4972-91ee-66f750490c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.428423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c71c5-bf71-4972-91ee-66f750490c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.429089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c71c5-bf71-4972-91ee-66f750490c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.436103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c71c5-bf71-4972-91ee-66f750490c1a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.441208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 925/1854] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:37.447231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c71c5-bf71-4972-91ee-66f750490c1a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:37.457034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c71c5-bf71-4972-91ee-66f750490c1a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.461664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70253061-4bef-4120-9549-0d3ac6c784ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.462969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70253061-4bef-4120-9549-0d3ac6c784ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.463351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70253061-4bef-4120-9549-0d3ac6c784ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.463642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70253061-4bef-4120-9549-0d3ac6c784ea admin admin] PATH_INFO: `/v3/projects/40978f4ba192486682faef328c1020f1/users/f0d87251d89e49c3869df4dbdd0e403c/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.481629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70253061-4bef-4120-9549-0d3ac6c784ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=40978f4ba192486682faef328c1020f1, user_id=f0d87251d89e49c3869df4dbdd0e403c, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.484440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70253061-4bef-4120-9549-0d3ac6c784ea admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.546039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 930/1855] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:37 2022] PUT /identity/v3/projects/419d159f16d645519676003ef0af8e6e/users/f1f654b76e8a4f1081800e0ddd3a552b/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:37.557354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a8314-4cdc-4d54-8654-9156cb4d202b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.557645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a8314-4cdc-4d54-8654-9156cb4d202b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.557915 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a8314-4cdc-4d54-8654-9156cb4d202b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.575407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 926/1856] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:37 2022] PUT /identity/v3/projects/40978f4ba192486682faef328c1020f1/users/f0d87251d89e49c3869df4dbdd0e403c/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:37.588324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f4468-0773-4b61-8927-2f0af386af8e None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.588594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f4468-0773-4b61-8927-2f0af386af8e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.588882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f4468-0773-4b61-8927-2f0af386af8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.590614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-cf4a8314-4cdc-4d54-8654-9156cb4d202b None None] MFA Rules not processed for user `f1f654b76e8a4f1081800e0ddd3a552b`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:37.591884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cf4a8314-4cdc-4d54-8654-9156cb4d202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:37.617534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6c0f4468-0773-4b61-8927-2f0af386af8e None None] MFA Rules not processed for user `f0d87251d89e49c3869df4dbdd0e403c`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:37.619378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6c0f4468-0773-4b61-8927-2f0af386af8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:37.659451 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 931/1857] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:37.663782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a8314-4cdc-4d54-8654-9156cb4d202b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:37.694246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 927/1858] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:37.706682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f4468-0773-4b61-8927-2f0af386af8e None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:37.711617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6c0f4468-0773-4b61-8927-2f0af386af8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:37.750672 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a8314-4cdc-4d54-8654-9156cb4d202b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.762722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc49009-65b3-45b0-9776-caaba2a77d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.763528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc49009-65b3-45b0-9776-caaba2a77d1a admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.764660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc49009-65b3-45b0-9776-caaba2a77d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.765064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc49009-65b3-45b0-9776-caaba2a77d1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.766078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc49009-65b3-45b0-9776-caaba2a77d1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.769092 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc49009-65b3-45b0-9776-caaba2a77d1a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.793913 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f4468-0773-4b61-8927-2f0af386af8e None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.798028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb285fa-f5ff-4e3f-8bc5-e6a3e1567c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.803313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb285fa-f5ff-4e3f-8bc5-e6a3e1567c0c service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.803313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb285fa-f5ff-4e3f-8bc5-e6a3e1567c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.803313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb285fa-f5ff-4e3f-8bc5-e6a3e1567c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.812309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb285fa-f5ff-4e3f-8bc5-e6a3e1567c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.813892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb285fa-f5ff-4e3f-8bc5-e6a3e1567c0c service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.825333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 928/1859] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:37.846175 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb285fa-f5ff-4e3f-8bc5-e6a3e1567c0c service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:37.860804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb285fa-f5ff-4e3f-8bc5-e6a3e1567c0c service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:37.866082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e7dfc-017d-4f60-8a3a-f6c1b11c0095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:37.866717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e7dfc-017d-4f60-8a3a-f6c1b11c0095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:37.867257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e7dfc-017d-4f60-8a3a-f6c1b11c0095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:37.868661 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e7dfc-017d-4f60-8a3a-f6c1b11c0095 admin admin] PATH_INFO: `/v3/users/f0d87251d89e49c3869df4dbdd0e403c` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:37.870087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e7dfc-017d-4f60-8a3a-f6c1b11c0095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d87251d89e49c3869df4dbdd0e403c)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:37.872152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e7dfc-017d-4f60-8a3a-f6c1b11c0095 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:37.954553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 932/1860] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:37 2022] POST /identity/v3/projects => generated 357 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:37.955735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc49009-65b3-45b0-9776-caaba2a77d1a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:38.039343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc49009-65b3-45b0-9776-caaba2a77d1a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:38.045036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e622d2-a577-4ff7-bf22-5fca11cc7f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:38.046688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e622d2-a577-4ff7-bf22-5fca11cc7f17 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:38.046688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e622d2-a577-4ff7-bf22-5fca11cc7f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:38.046963 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e622d2-a577-4ff7-bf22-5fca11cc7f17 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:38.047555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e622d2-a577-4ff7-bf22-5fca11cc7f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:38.049636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e622d2-a577-4ff7-bf22-5fca11cc7f17 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:38.274060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 933/1861] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:37 2022] POST /identity/v3/users => generated 285 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:38.275381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e622d2-a577-4ff7-bf22-5fca11cc7f17 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:38.297893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e622d2-a577-4ff7-bf22-5fca11cc7f17 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:38.317701 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa987a-8926-4d08-a76e-962e3e19fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:38.318336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa987a-8926-4d08-a76e-962e3e19fbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:38.318622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa987a-8926-4d08-a76e-962e3e19fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:38.318901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa987a-8926-4d08-a76e-962e3e19fbc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:38.320063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa987a-8926-4d08-a76e-962e3e19fbc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:38.321387 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa987a-8926-4d08-a76e-962e3e19fbc4 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:38.337847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 934/1862] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:38.338566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa987a-8926-4d08-a76e-962e3e19fbc4 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:38.341519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-650e7dfc-017d-4f60-8a3a-f6c1b11c0095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d87251d89e49c3869df4dbdd0e403c'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:38.342574 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-650e7dfc-017d-4f60-8a3a-f6c1b11c0095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d87251d89e49c3869df4dbdd0e403c'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:38.350799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-650e7dfc-017d-4f60-8a3a-f6c1b11c0095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d87251d89e49c3869df4dbdd0e403c'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:38.353167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-650e7dfc-017d-4f60-8a3a-f6c1b11c0095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d87251d89e49c3869df4dbdd0e403c'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:38.357557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa987a-8926-4d08-a76e-962e3e19fbc4 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:38.366370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761006e-d62f-437e-9a85-d9f4e0411f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:38.368060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761006e-d62f-437e-9a85-d9f4e0411f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:38.369078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761006e-d62f-437e-9a85-d9f4e0411f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:38.369792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761006e-d62f-437e-9a85-d9f4e0411f0c admin admin] PATH_INFO: `/v3/projects/b5fb56c3ecb940099d3121bdeb7fc984/users/a86cf31df3a94c63acb71852c73ce192/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:38.394645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761006e-d62f-437e-9a85-d9f4e0411f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5fb56c3ecb940099d3121bdeb7fc984, user_id=a86cf31df3a94c63acb71852c73ce192, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:38.397806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761006e-d62f-437e-9a85-d9f4e0411f0c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:38.441775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-650e7dfc-017d-4f60-8a3a-f6c1b11c0095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d87251d89e49c3869df4dbdd0e403c'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:38.470037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 929/1863] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:37 2022] DELETE /identity/v3/users/f0d87251d89e49c3869df4dbdd0e403c => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:38.474937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ea943-3bb0-40fa-b724-a39b85f6080c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:38.480335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ea943-3bb0-40fa-b724-a39b85f6080c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:38.480335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ea943-3bb0-40fa-b724-a39b85f6080c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:38.504473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-007ea943-3bb0-40fa-b724-a39b85f6080c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:38.506170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-007ea943-3bb0-40fa-b724-a39b85f6080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:38.566542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 935/1864] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:38 2022] PUT /identity/v3/projects/b5fb56c3ecb940099d3121bdeb7fc984/users/a86cf31df3a94c63acb71852c73ce192/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:38.570913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4257f-bb16-4dae-8d9f-c9e78b7f2f4b None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:38.572090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4257f-bb16-4dae-8d9f-c9e78b7f2f4b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:38.573829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4257f-bb16-4dae-8d9f-c9e78b7f2f4b None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:38.579437 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 936/1865] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:17:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:17:38.581283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4257f-bb16-4dae-8d9f-c9e78b7f2f4b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:38.589619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-07b4257f-bb16-4dae-8d9f-c9e78b7f2f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:38.615610 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 930/1866] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:38.617946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9a41-ca0d-4113-a2e1-0844a60580b2 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:38.619171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9a41-ca0d-4113-a2e1-0844a60580b2 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:38.619694 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9a41-ca0d-4113-a2e1-0844a60580b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:38.657334 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-033d9a41-ca0d-4113-a2e1-0844a60580b2 None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:38.659250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-033d9a41-ca0d-4113-a2e1-0844a60580b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:38.810384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4257f-bb16-4dae-8d9f-c9e78b7f2f4b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:38.818962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926f0bc-528a-402f-8b44-a98476290e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:38.820381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926f0bc-528a-402f-8b44-a98476290e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:38.820674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926f0bc-528a-402f-8b44-a98476290e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:38.825341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926f0bc-528a-402f-8b44-a98476290e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:38.825341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926f0bc-528a-402f-8b44-a98476290e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:38.825341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926f0bc-528a-402f-8b44-a98476290e36 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:38.838843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 937/1867] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:38.845064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926f0bc-528a-402f-8b44-a98476290e36 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:38.851740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b926f0bc-528a-402f-8b44-a98476290e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:38.881860 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 931/1868] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:17:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:38.882723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d9a41-ca0d-4113-a2e1-0844a60580b2 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:38.906084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d9a41-ca0d-4113-a2e1-0844a60580b2 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:38.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd886b8b-539e-4b17-b1bf-7eb1ce904dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:38.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd886b8b-539e-4b17-b1bf-7eb1ce904dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:38.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd886b8b-539e-4b17-b1bf-7eb1ce904dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:38.923119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd886b8b-539e-4b17-b1bf-7eb1ce904dc4 admin admin] PATH_INFO: `/v3/projects/b5fb56c3ecb940099d3121bdeb7fc984/users/a86cf31df3a94c63acb71852c73ce192/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:38.923412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd886b8b-539e-4b17-b1bf-7eb1ce904dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5fb56c3ecb940099d3121bdeb7fc984, user_id=a86cf31df3a94c63acb71852c73ce192, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:38.924026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd886b8b-539e-4b17-b1bf-7eb1ce904dc4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:38.951499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 932/1869] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:38 2022] PUT /identity/v3/projects/b5fb56c3ecb940099d3121bdeb7fc984/users/a86cf31df3a94c63acb71852c73ce192/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:38.952381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd886b8b-539e-4b17-b1bf-7eb1ce904dc4 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:38.953670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926f0bc-528a-402f-8b44-a98476290e36 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:38.954165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-bd886b8b-539e-4b17-b1bf-7eb1ce904dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:39.021439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85a098-f3ee-4d37-9b5c-583b073fed3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:39.031957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a098-f3ee-4d37-9b5c-583b073fed3d service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:39.031957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a098-f3ee-4d37-9b5c-583b073fed3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:39.031957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a098-f3ee-4d37-9b5c-583b073fed3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:39.062400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd886b8b-539e-4b17-b1bf-7eb1ce904dc4 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:39.069291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92783c07-e0e3-4e66-8cc8-701122dc8f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:39.070202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92783c07-e0e3-4e66-8cc8-701122dc8f3e service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:39.070689 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92783c07-e0e3-4e66-8cc8-701122dc8f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:39.071598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92783c07-e0e3-4e66-8cc8-701122dc8f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:39.074731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-92783c07-e0e3-4e66-8cc8-701122dc8f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:39.121296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85a098-f3ee-4d37-9b5c-583b073fed3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:39.123298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85a098-f3ee-4d37-9b5c-583b073fed3d service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:39.146667 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 938/1870] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:39.149903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85a098-f3ee-4d37-9b5c-583b073fed3d service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:39.153992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5f85a098-f3ee-4d37-9b5c-583b073fed3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:39.192271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92783c07-e0e3-4e66-8cc8-701122dc8f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:39.194880 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92783c07-e0e3-4e66-8cc8-701122dc8f3e service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:39.216735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 933/1871] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:39.217727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92783c07-e0e3-4e66-8cc8-701122dc8f3e service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:39.219407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-92783c07-e0e3-4e66-8cc8-701122dc8f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:39.237443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92783c07-e0e3-4e66-8cc8-701122dc8f3e service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:39.248059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:39.248886 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:39.249237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:39.249847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] PATH_INFO: `/v3/users/af97b440e72545ce9b654148966ea4a2` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:39.251410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=af97b440e72545ce9b654148966ea4a2)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:39.253851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:39.317171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85a098-f3ee-4d37-9b5c-583b073fed3d service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:39.324315 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aed6b-297d-42ec-bd3c-3c0e840a53b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:39.325321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aed6b-297d-42ec-bd3c-3c0e840a53b5 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:39.325698 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aed6b-297d-42ec-bd3c-3c0e840a53b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:39.326041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aed6b-297d-42ec-bd3c-3c0e840a53b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:39.337431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91aed6b-297d-42ec-bd3c-3c0e840a53b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:39.337431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91aed6b-297d-42ec-bd3c-3c0e840a53b5 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:39.354590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 939/1872] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:39.356590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9419268-0b8b-4800-8857-3b422b1dd8e5 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:39.356911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9419268-0b8b-4800-8857-3b422b1dd8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:39.357344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9419268-0b8b-4800-8857-3b422b1dd8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:39.393014 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-f9419268-0b8b-4800-8857-3b422b1dd8e5 None None] MFA Rules not processed for user `a86cf31df3a94c63acb71852c73ce192`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:39.394405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f9419268-0b8b-4800-8857-3b422b1dd8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:39.514451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af97b440e72545ce9b654148966ea4a2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:39.515766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af97b440e72545ce9b654148966ea4a2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:39.520956 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 940/1873] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:39 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:39.533898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9419268-0b8b-4800-8857-3b422b1dd8e5 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:39.536011 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af97b440e72545ce9b654148966ea4a2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:39.536706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af97b440e72545ce9b654148966ea4a2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:39.546517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9419268-0b8b-4800-8857-3b422b1dd8e5 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:39.550895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f41411-c4a7-4c4d-82a2-0fc483618f1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:39.551822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41411-c4a7-4c4d-82a2-0fc483618f1d service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:39.552172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41411-c4a7-4c4d-82a2-0fc483618f1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:39.552574 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41411-c4a7-4c4d-82a2-0fc483618f1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:39.554630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-62f41411-c4a7-4c4d-82a2-0fc483618f1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:39.567885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f41411-c4a7-4c4d-82a2-0fc483618f1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:39.569489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f41411-c4a7-4c4d-82a2-0fc483618f1d service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:39.580633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 941/1874] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:39 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:39.633482 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af97b440e72545ce9b654148966ea4a2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:39.661963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 934/1875] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:39 2022] DELETE /identity/v3/users/af97b440e72545ce9b654148966ea4a2 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:39.667323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbce0a-9732-4273-a89e-85db7dc82543 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:39.667697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbce0a-9732-4273-a89e-85db7dc82543 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:39.668039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbce0a-9732-4273-a89e-85db7dc82543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:39.689660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-dfcbce0a-9732-4273-a89e-85db7dc82543 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:39.690979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-dfcbce0a-9732-4273-a89e-85db7dc82543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:39.776412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 942/1876] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:39.788103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:39.788652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:39.804085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbce0a-9732-4273-a89e-85db7dc82543 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:39.806348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-dfcbce0a-9732-4273-a89e-85db7dc82543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:39.828712 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbce0a-9732-4273-a89e-85db7dc82543 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:39.834275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:39.835108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:39.835544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:39.835922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] PATH_INFO: `/v3/projects/40978f4ba192486682faef328c1020f1` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:39.838079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=40978f4ba192486682faef328c1020f1)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:39.840316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:39.985252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62c956-6cb3-4354-9c2a-ce55eb7c4b6a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.000895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40978f4ba192486682faef328c1020f1', 'request_id': 'req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:40.008361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40978f4ba192486682faef328c1020f1', 'request_id': 'req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:40.058148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb9a23-0e0b-451b-963c-05859ba05f20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.058812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb9a23-0e0b-451b-963c-05859ba05f20 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.059100 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb9a23-0e0b-451b-963c-05859ba05f20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.059374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb9a23-0e0b-451b-963c-05859ba05f20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.095263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40978f4ba192486682faef328c1020f1', 'request_id': 'req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:40.124398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] The token cache is being invalidate because project 40978f4ba192486682faef328c1020f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:40.124706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:40.125441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:40.133441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 943/1877] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:39 2022] DELETE /identity/v3/projects/40978f4ba192486682faef328c1020f1 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:40.133441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:40.138206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:40.217412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb9a23-0e0b-451b-963c-05859ba05f20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.219084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb9a23-0e0b-451b-963c-05859ba05f20 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.231754 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 935/1878] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:40.233061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0770-e0da-4090-96b3-b84d5be4e6c7 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.233346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0770-e0da-4090-96b3-b84d5be4e6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.233617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0770-e0da-4090-96b3-b84d5be4e6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.234570 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290b8cf-c5a7-4ab9-b0a6-49b5a25f9eef admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.240166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017be3ba-63df-4e83-942f-cb64a6c94cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.240803 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017be3ba-63df-4e83-942f-cb64a6c94cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.241290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017be3ba-63df-4e83-942f-cb64a6c94cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.241672 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017be3ba-63df-4e83-942f-cb64a6c94cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.244895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-017be3ba-63df-4e83-942f-cb64a6c94cf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:40.261016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-2ccd0770-e0da-4090-96b3-b84d5be4e6c7 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:40.262767 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2ccd0770-e0da-4090-96b3-b84d5be4e6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:40.271700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 936/1879] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:40.280985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ebe82-d66c-4bd2-b718-f652a8041b08 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.285053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ebe82-d66c-4bd2-b718-f652a8041b08 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.285053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ebe82-d66c-4bd2-b718-f652a8041b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.311911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-7e1ebe82-d66c-4bd2-b718-f652a8041b08 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:40.314112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7e1ebe82-d66c-4bd2-b718-f652a8041b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:40.323947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017be3ba-63df-4e83-942f-cb64a6c94cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.325585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017be3ba-63df-4e83-942f-cb64a6c94cf6 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.325816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 937/1880] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:40.329603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ebe82-d66c-4bd2-b718-f652a8041b08 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:40.335530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 944/1881] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:40.354193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ebe82-d66c-4bd2-b718-f652a8041b08 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.359121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be7d03-d1e5-43bc-abc9-c705c14df2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.359791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be7d03-d1e5-43bc-abc9-c705c14df2cc admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.360073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be7d03-d1e5-43bc-abc9-c705c14df2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.360387 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be7d03-d1e5-43bc-abc9-c705c14df2cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.361015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be7d03-d1e5-43bc-abc9-c705c14df2cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:40.361332 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be7d03-d1e5-43bc-abc9-c705c14df2cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.362889 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be7d03-d1e5-43bc-abc9-c705c14df2cc admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.377485 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 938/1882] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:17:40.381679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017be3ba-63df-4e83-942f-cb64a6c94cf6 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:40.393657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017be3ba-63df-4e83-942f-cb64a6c94cf6 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.398409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f3bd5-bae8-46a8-9fad-c9c03076e907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.399037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f3bd5-bae8-46a8-9fad-c9c03076e907 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.399568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f3bd5-bae8-46a8-9fad-c9c03076e907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.399768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f3bd5-bae8-46a8-9fad-c9c03076e907 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.400637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f3bd5-bae8-46a8-9fad-c9c03076e907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.403178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f3bd5-bae8-46a8-9fad-c9c03076e907 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.508711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 945/1883] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:40 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:40.510449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be7d03-d1e5-43bc-abc9-c705c14df2cc admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:40.521006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be7d03-d1e5-43bc-abc9-c705c14df2cc admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.541429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f3bd5-bae8-46a8-9fad-c9c03076e907 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:40.543337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9e7f3bd5-bae8-46a8-9fad-c9c03076e907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:40.596146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f11e84-553a-4619-a75d-7846b9738e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.597059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f11e84-553a-4619-a75d-7846b9738e6d admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.597440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f11e84-553a-4619-a75d-7846b9738e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.597815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f11e84-553a-4619-a75d-7846b9738e6d admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.598933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f11e84-553a-4619-a75d-7846b9738e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.601940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f11e84-553a-4619-a75d-7846b9738e6d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.629453 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f3bd5-bae8-46a8-9fad-c9c03076e907 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.635660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70f620-470c-4e55-a30a-8e36f2731a88 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e4555deb6ce44b382c12061be7c7de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a193770d410416a944a802a1407304e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.648064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70f620-470c-4e55-a30a-8e36f2731a88 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.648064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70f620-470c-4e55-a30a-8e36f2731a88 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.648064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70f620-470c-4e55-a30a-8e36f2731a88 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.648064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70f620-470c-4e55-a30a-8e36f2731a88 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:40.648064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70f620-470c-4e55-a30a-8e36f2731a88 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.649068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70f620-470c-4e55-a30a-8e36f2731a88 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.659459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 946/1884] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:40.664132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70f620-470c-4e55-a30a-8e36f2731a88 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:40.679834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70f620-470c-4e55-a30a-8e36f2731a88 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.686392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8e42e-0cbd-46cd-a770-4cb6cc079e5d tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e4555deb6ce44b382c12061be7c7de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a193770d410416a944a802a1407304e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.693937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8e42e-0cbd-46cd-a770-4cb6cc079e5d tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.693937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8e42e-0cbd-46cd-a770-4cb6cc079e5d tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.693937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8e42e-0cbd-46cd-a770-4cb6cc079e5d tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.693937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8e42e-0cbd-46cd-a770-4cb6cc079e5d tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.693937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8e42e-0cbd-46cd-a770-4cb6cc079e5d tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.769159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 939/1885] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:40 2022] POST /identity/v3/users => generated 292 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:40.772791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f11e84-553a-4619-a75d-7846b9738e6d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:40.784149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f11e84-553a-4619-a75d-7846b9738e6d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.789042 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b44ff0-b9ce-4d04-a3b2-807512374a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.789642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b44ff0-b9ce-4d04-a3b2-807512374a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.789996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b44ff0-b9ce-4d04-a3b2-807512374a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.790499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b44ff0-b9ce-4d04-a3b2-807512374a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.791152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b44ff0-b9ce-4d04-a3b2-807512374a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.792746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b44ff0-b9ce-4d04-a3b2-807512374a70 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.794554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:40.807296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 940/1886] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:40.808761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b44ff0-b9ce-4d04-a3b2-807512374a70 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:40.812277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 947/1887] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:40 2022] POST /identity/v3/projects => generated 354 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:40.813460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8e42e-0cbd-46cd-a770-4cb6cc079e5d tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:40.823867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b44ff0-b9ce-4d04-a3b2-807512374a70 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.889304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8e42e-0cbd-46cd-a770-4cb6cc079e5d tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.889657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c17eb-e78e-43e4-8be4-ab81843e1031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.890264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c17eb-e78e-43e4-8be4-ab81843e1031 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.890547 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c17eb-e78e-43e4-8be4-ab81843e1031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.890829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c17eb-e78e-43e4-8be4-ab81843e1031 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.891582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c17eb-e78e-43e4-8be4-ab81843e1031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.893214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c17eb-e78e-43e4-8be4-ab81843e1031 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.894443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36537315-c722-42f2-a73b-1e2d080d22cb tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e4555deb6ce44b382c12061be7c7de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a193770d410416a944a802a1407304e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.895061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36537315-c722-42f2-a73b-1e2d080d22cb tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.895377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36537315-c722-42f2-a73b-1e2d080d22cb tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.895654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36537315-c722-42f2-a73b-1e2d080d22cb tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.896305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36537315-c722-42f2-a73b-1e2d080d22cb tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:40.896705 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36537315-c722-42f2-a73b-1e2d080d22cb tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.898345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36537315-c722-42f2-a73b-1e2d080d22cb tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.903211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 941/1888] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:40.907553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c17eb-e78e-43e4-8be4-ab81843e1031 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:40.913704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 948/1889] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:40.919781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c17eb-e78e-43e4-8be4-ab81843e1031 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.924726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c088257-6924-4c08-8207-f60769510143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.925676 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c088257-6924-4c08-8207-f60769510143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.925960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c088257-6924-4c08-8207-f60769510143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.926237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c088257-6924-4c08-8207-f60769510143 admin admin] PATH_INFO: `/v3/projects/d55b789674d5481f97fd3cb33fdc3f40/users/07506462884949f1839d1e9dcc86e513/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.946730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c088257-6924-4c08-8207-f60769510143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d55b789674d5481f97fd3cb33fdc3f40, user_id=07506462884949f1839d1e9dcc86e513, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.949996 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c088257-6924-4c08-8207-f60769510143 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.960798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36537315-c722-42f2-a73b-1e2d080d22cb tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:40.970732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36537315-c722-42f2-a73b-1e2d080d22cb tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:40.981840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e4555deb6ce44b382c12061be7c7de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a193770d410416a944a802a1407304e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:40.982564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:40.982853 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:40.983220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] PATH_INFO: `/v3/projects/cd854c247eeb435e937dcb6b3fddef52` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:40.984374 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cd854c247eeb435e937dcb6b3fddef52)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:40.986472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:40.992155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:41.113292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 942/1890] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:40 2022] PUT /identity/v3/projects/d55b789674d5481f97fd3cb33fdc3f40/users/07506462884949f1839d1e9dcc86e513/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:41.140793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c088257-6924-4c08-8207-f60769510143 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:41.147700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd854c247eeb435e937dcb6b3fddef52', 'request_id': 'req-59413612-0e90-499b-bad7-de4b95a487d9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:41.147933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2c088257-6924-4c08-8207-f60769510143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:41.148024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd854c247eeb435e937dcb6b3fddef52', 'request_id': 'req-59413612-0e90-499b-bad7-de4b95a487d9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:41.236832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd854c247eeb435e937dcb6b3fddef52', 'request_id': 'req-59413612-0e90-499b-bad7-de4b95a487d9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:41.252354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c088257-6924-4c08-8207-f60769510143 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:41.261837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] The token cache is being invalidate because project cd854c247eeb435e937dcb6b3fddef52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:41.270188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:41.270188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-59413612-0e90-499b-bad7-de4b95a487d9 tempest-VolumeQuotaClassesTest-173263454 tempest-VolumeQuotaClassesTest-173263454-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:41.270188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 949/1891] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:40 2022] DELETE /identity/v3/projects/cd854c247eeb435e937dcb6b3fddef52 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:41.272615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec3d67-8285-4275-8849-bdbe556b1b47 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:41.274219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec3d67-8285-4275-8849-bdbe556b1b47 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:41.274843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec3d67-8285-4275-8849-bdbe556b1b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:41.323322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-60ec3d67-8285-4275-8849-bdbe556b1b47 None None] MFA Rules not processed for user `07506462884949f1839d1e9dcc86e513`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:41.325308 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-60ec3d67-8285-4275-8849-bdbe556b1b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:41.380081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:41.381034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:41.381499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:41.381932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] PATH_INFO: `/v3/projects/4ead214ac2af4ac28b4d81d3ad0a61f4` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:41.384094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ead214ac2af4ac28b4d81d3ad0a61f4)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:41.387006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:41.421308 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 950/1892] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:41.435065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec3d67-8285-4275-8849-bdbe556b1b47 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:41.436571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-60ec3d67-8285-4275-8849-bdbe556b1b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:41.504535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec3d67-8285-4275-8849-bdbe556b1b47 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:41.509151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcbf5d-34f3-465f-babc-54be2a2b59c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:41.510059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcbf5d-34f3-465f-babc-54be2a2b59c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:41.510409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcbf5d-34f3-465f-babc-54be2a2b59c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:41.510761 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcbf5d-34f3-465f-babc-54be2a2b59c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:41.520141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcbf5d-34f3-465f-babc-54be2a2b59c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:41.532202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcbf5d-34f3-465f-babc-54be2a2b59c3 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:41.547468 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 951/1893] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:41.565206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ead214ac2af4ac28b4d81d3ad0a61f4', 'request_id': 'req-a0d664ea-a786-4055-9771-83be285b97bc'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:41.575126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ead214ac2af4ac28b4d81d3ad0a61f4', 'request_id': 'req-a0d664ea-a786-4055-9771-83be285b97bc'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:41.590177 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ead214ac2af4ac28b4d81d3ad0a61f4', 'request_id': 'req-a0d664ea-a786-4055-9771-83be285b97bc'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:41.681064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] The token cache is being invalidate because project 4ead214ac2af4ac28b4d81d3ad0a61f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:41.681385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:41.682229 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:41.684306 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 943/1894] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:41 2022] DELETE /identity/v3/projects/4ead214ac2af4ac28b4d81d3ad0a61f4 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:41.688298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcbf5d-34f3-465f-babc-54be2a2b59c3 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:41.689870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d8dcbf5d-34f3-465f-babc-54be2a2b59c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:41.708506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcbf5d-34f3-465f-babc-54be2a2b59c3 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:41.786995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae8d23-c098-4450-9dd0-0a47100693d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:41.787944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae8d23-c098-4450-9dd0-0a47100693d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:41.788230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae8d23-c098-4450-9dd0-0a47100693d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:41.788508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae8d23-c098-4450-9dd0-0a47100693d9 admin admin] PATH_INFO: `/v3/users/31704c3041a145408693f663ebb7c616` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:41.790406 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae8d23-c098-4450-9dd0-0a47100693d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31704c3041a145408693f663ebb7c616)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:41.792602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae8d23-c098-4450-9dd0-0a47100693d9 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:42.064488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-fdae8d23-c098-4450-9dd0-0a47100693d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31704c3041a145408693f663ebb7c616'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:42.065241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-fdae8d23-c098-4450-9dd0-0a47100693d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31704c3041a145408693f663ebb7c616'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:42.072666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-fdae8d23-c098-4450-9dd0-0a47100693d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31704c3041a145408693f663ebb7c616'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:42.092903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-fdae8d23-c098-4450-9dd0-0a47100693d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31704c3041a145408693f663ebb7c616'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:42.139728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:42.142458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:42.154967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d664ea-a786-4055-9771-83be285b97bc admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:42.166703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:42.166703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:42.166703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:42.166703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] PATH_INFO: `/v3/users/7caa5b45f57e443a8d3bfe24cfd145d3` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:42.167050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7caa5b45f57e443a8d3bfe24cfd145d3)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:42.167050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:42.257026 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-fdae8d23-c098-4450-9dd0-0a47100693d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31704c3041a145408693f663ebb7c616'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:42.268581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 952/1895] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:41 2022] DELETE /identity/v3/users/31704c3041a145408693f663ebb7c616 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:42.590522 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7caa5b45f57e443a8d3bfe24cfd145d3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:42.592189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7caa5b45f57e443a8d3bfe24cfd145d3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:42.599647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7caa5b45f57e443a8d3bfe24cfd145d3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:42.600455 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7caa5b45f57e443a8d3bfe24cfd145d3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:42.705901 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7caa5b45f57e443a8d3bfe24cfd145d3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:42.736506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 944/1896] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:42 2022] DELETE /identity/v3/users/7caa5b45f57e443a8d3bfe24cfd145d3 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:42.739177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473ce3a-8d1e-44ff-b458-12cda16407d0 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:42.739177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473ce3a-8d1e-44ff-b458-12cda16407d0 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:42.739740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473ce3a-8d1e-44ff-b458-12cda16407d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:42.766473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-5473ce3a-8d1e-44ff-b458-12cda16407d0 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:42.767786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5473ce3a-8d1e-44ff-b458-12cda16407d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:42.844463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 953/1897] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:42.853855 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:42.855304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:42.920804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82032a54-4d13-42b4-a2f1-0d3d6ede3e03 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:42.937931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec935d3-303b-4a29-970e-096138a48845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:42.937931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec935d3-303b-4a29-970e-096138a48845 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:42.937931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec935d3-303b-4a29-970e-096138a48845 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:42.937931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec935d3-303b-4a29-970e-096138a48845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:42.941930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec935d3-303b-4a29-970e-096138a48845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:42.943466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec935d3-303b-4a29-970e-096138a48845 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:42.956500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 945/1898] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:42.986447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473ce3a-8d1e-44ff-b458-12cda16407d0 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:42.988688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5473ce3a-8d1e-44ff-b458-12cda16407d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:43.008343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473ce3a-8d1e-44ff-b458-12cda16407d0 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:43.016299 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:43.017039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:43.017343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:43.017603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] PATH_INFO: `/v3/projects/9662704043f641f4bed814c8e438ac73` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:43.018842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9662704043f641f4bed814c8e438ac73)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:43.021002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:43.147815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9662704043f641f4bed814c8e438ac73', 'request_id': 'req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.155944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9662704043f641f4bed814c8e438ac73', 'request_id': 'req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.231635 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9662704043f641f4bed814c8e438ac73', 'request_id': 'req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.245150 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] The token cache is being invalidate because project 9662704043f641f4bed814c8e438ac73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:43.245798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.246481 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.248137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 954/1899] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:42 2022] DELETE /identity/v3/projects/9662704043f641f4bed814c8e438ac73 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:43.252518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec935d3-303b-4a29-970e-096138a48845 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:43.277703 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec935d3-303b-4a29-970e-096138a48845 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:43.343135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:43.373091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:43.373884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:43.374171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:43.374807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] PATH_INFO: `/v3/users/712ca787a4784e6b8df64cb2004ac964` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:43.376312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=712ca787a4784e6b8df64cb2004ac964)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:43.377762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:43.424231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:43.565304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:43.566733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:43.582106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9f039-44b5-48ea-b72c-7b0cc2f8adf4 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:43.589714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:43.590865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:43.591206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:43.591505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] PATH_INFO: `/v3/projects/496cf4043b844c76a9c2d389b74cf172` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:43.593032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=496cf4043b844c76a9c2d389b74cf172)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:43.595026 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:43.665406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '712ca787a4784e6b8df64cb2004ac964'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.666430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '712ca787a4784e6b8df64cb2004ac964'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.673952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '712ca787a4784e6b8df64cb2004ac964'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.674696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '712ca787a4784e6b8df64cb2004ac964'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.746571 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '712ca787a4784e6b8df64cb2004ac964'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.767425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '496cf4043b844c76a9c2d389b74cf172', 'request_id': 'req-410c4914-6cad-4c31-875b-9fbc41a31e0f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.771789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 946/1900] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:43 2022] DELETE /identity/v3/users/712ca787a4784e6b8df64cb2004ac964 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:43.781584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '496cf4043b844c76a9c2d389b74cf172', 'request_id': 'req-410c4914-6cad-4c31-875b-9fbc41a31e0f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.911861 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '496cf4043b844c76a9c2d389b74cf172', 'request_id': 'req-410c4914-6cad-4c31-875b-9fbc41a31e0f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.935125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] The token cache is being invalidate because project 496cf4043b844c76a9c2d389b74cf172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:43.935398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.936292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:43.938138 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 955/1901] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:43 2022] DELETE /identity/v3/projects/496cf4043b844c76a9c2d389b74cf172 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:43.942578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:43.948993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:44.083070 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd100799-3ecc-42fc-b67c-73601ecc4b7e admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:44.099645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dac01-eeee-485b-a7e2-15b796b2cda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:44.111408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dac01-eeee-485b-a7e2-15b796b2cda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:44.111712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dac01-eeee-485b-a7e2-15b796b2cda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:44.111998 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dac01-eeee-485b-a7e2-15b796b2cda1 admin admin] PATH_INFO: `/v3/users/0e4555deb6ce44b382c12061be7c7de3` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:44.120106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dac01-eeee-485b-a7e2-15b796b2cda1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e4555deb6ce44b382c12061be7c7de3)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:44.122565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dac01-eeee-485b-a7e2-15b796b2cda1 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:44.403825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-673dac01-eeee-485b-a7e2-15b796b2cda1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e4555deb6ce44b382c12061be7c7de3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:44.404487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-673dac01-eeee-485b-a7e2-15b796b2cda1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e4555deb6ce44b382c12061be7c7de3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:44.412248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-673dac01-eeee-485b-a7e2-15b796b2cda1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e4555deb6ce44b382c12061be7c7de3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:44.413075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-673dac01-eeee-485b-a7e2-15b796b2cda1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e4555deb6ce44b382c12061be7c7de3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:44.456517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:44.458429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:44.518726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-673dac01-eeee-485b-a7e2-15b796b2cda1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e4555deb6ce44b382c12061be7c7de3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:44.553541 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 947/1902] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:43 2022] DELETE /identity/v3/users/0e4555deb6ce44b382c12061be7c7de3 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:44.578474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c4914-6cad-4c31-875b-9fbc41a31e0f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:44.675244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:44.675945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:44.676230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:44.676523 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] PATH_INFO: `/v3/projects/0070db64a75f4bf6b421779f0bbd0a83` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:44.677786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] RBAC: Authorizing `identity:delete_project(project_id=0070db64a75f4bf6b421779f0bbd0a83)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:44.679665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:44.749685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:44.852081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0070db64a75f4bf6b421779f0bbd0a83', 'request_id': 'req-3c3d8e61-97df-4e94-9def-d9c8dcccacce'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:44.857568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0070db64a75f4bf6b421779f0bbd0a83', 'request_id': 'req-3c3d8e61-97df-4e94-9def-d9c8dcccacce'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:44.873611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:44.949416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0070db64a75f4bf6b421779f0bbd0a83', 'request_id': 'req-3c3d8e61-97df-4e94-9def-d9c8dcccacce'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:44.964816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] The token cache is being invalidate because project 0070db64a75f4bf6b421779f0bbd0a83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:44.965372 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:44.974963 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3c3d8e61-97df-4e94-9def-d9c8dcccacce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:44.978649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 956/1903] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:44 2022] DELETE /identity/v3/projects/0070db64a75f4bf6b421779f0bbd0a83 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:44.984055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b480296-8bc3-4f04-9a4c-c0c4347282dd None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:44.984585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b480296-8bc3-4f04-9a4c-c0c4347282dd None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:44.985352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b480296-8bc3-4f04-9a4c-c0c4347282dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.027418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6b480296-8bc3-4f04-9a4c-c0c4347282dd None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:45.028672 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6b480296-8bc3-4f04-9a4c-c0c4347282dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:45.089142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 948/1904] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:45.097400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13eb85-f45a-4b98-ad5d-df1569a13a3d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.097715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13eb85-f45a-4b98-ad5d-df1569a13a3d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.097992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13eb85-f45a-4b98-ad5d-df1569a13a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.118758 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-6b13eb85-f45a-4b98-ad5d-df1569a13a3d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:45.119955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6b13eb85-f45a-4b98-ad5d-df1569a13a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:45.127837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 957/1905] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:45.132047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b480296-8bc3-4f04-9a4c-c0c4347282dd None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:45.143090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b480296-8bc3-4f04-9a4c-c0c4347282dd None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:45.148183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea8171-9f48-42aa-96c2-89b2689a72ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:45.148816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea8171-9f48-42aa-96c2-89b2689a72ff admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.149974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea8171-9f48-42aa-96c2-89b2689a72ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.149974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea8171-9f48-42aa-96c2-89b2689a72ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.150787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea8171-9f48-42aa-96c2-89b2689a72ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:45.150787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea8171-9f48-42aa-96c2-89b2689a72ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:45.152259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea8171-9f48-42aa-96c2-89b2689a72ff admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:45.164358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 949/1906] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:45.169066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13eb85-f45a-4b98-ad5d-df1569a13a3d None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:45.178712 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13eb85-f45a-4b98-ad5d-df1569a13a3d None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:45.184760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea8171-9f48-42aa-96c2-89b2689a72ff admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:45.186853 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f9ea8171-9f48-42aa-96c2-89b2689a72ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:45.189524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c57df-4aca-41d1-bc18-6715ce02873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:45.190147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c57df-4aca-41d1-bc18-6715ce02873a admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.190425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c57df-4aca-41d1-bc18-6715ce02873a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.190696 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c57df-4aca-41d1-bc18-6715ce02873a admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.191325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c57df-4aca-41d1-bc18-6715ce02873a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:45.193404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c57df-4aca-41d1-bc18-6715ce02873a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:45.209481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea8171-9f48-42aa-96c2-89b2689a72ff admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:45.215671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:45.216585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.216874 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.217267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] PATH_INFO: `/v3/projects/7a193770d410416a944a802a1407304e` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.218967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a193770d410416a944a802a1407304e)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:45.220921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:45.303611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 958/1907] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:45 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:45.308805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c57df-4aca-41d1-bc18-6715ce02873a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:45.341035 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a193770d410416a944a802a1407304e', 'request_id': 'req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:45.346690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a193770d410416a944a802a1407304e', 'request_id': 'req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:45.353906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a193770d410416a944a802a1407304e', 'request_id': 'req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:45.378797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c57df-4aca-41d1-bc18-6715ce02873a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:45.438203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587009a1-f45c-4144-b3ef-1e4306eca9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:45.438826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587009a1-f45c-4144-b3ef-1e4306eca9cf admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.439124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587009a1-f45c-4144-b3ef-1e4306eca9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.439390 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587009a1-f45c-4144-b3ef-1e4306eca9cf admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.440023 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587009a1-f45c-4144-b3ef-1e4306eca9cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:45.442033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587009a1-f45c-4144-b3ef-1e4306eca9cf admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:45.459671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] The token cache is being invalidate because project 7a193770d410416a944a802a1407304e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:45.459968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:45.460629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ebe9d59c-4e11-44d6-98e1-70a454e6bb76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:45.462455 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 950/1908] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:45 2022] DELETE /identity/v3/projects/7a193770d410416a944a802a1407304e => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:45.467169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6535e0-9d9a-4350-b3cd-200f6d2a73d3 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.467451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6535e0-9d9a-4350-b3cd-200f6d2a73d3 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.467750 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6535e0-9d9a-4350-b3cd-200f6d2a73d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.487813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6c6535e0-9d9a-4350-b3cd-200f6d2a73d3 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:45.488973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6c6535e0-9d9a-4350-b3cd-200f6d2a73d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:45.498208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 951/1909] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:45.506027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4db643-97a4-468d-b3cb-491dc1705d15 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.506317 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4db643-97a4-468d-b3cb-491dc1705d15 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.506596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4db643-97a4-468d-b3cb-491dc1705d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.526440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-ec4db643-97a4-468d-b3cb-491dc1705d15 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:45.527653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ec4db643-97a4-468d-b3cb-491dc1705d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:45.535682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 952/1910] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:45.540221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4db643-97a4-468d-b3cb-491dc1705d15 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:45.549250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4db643-97a4-468d-b3cb-491dc1705d15 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:45.553828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f8b9f-2544-4966-8d94-bd9e9aecfa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:45.554393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f8b9f-2544-4966-8d94-bd9e9aecfa8f admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.554669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f8b9f-2544-4966-8d94-bd9e9aecfa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.557102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f8b9f-2544-4966-8d94-bd9e9aecfa8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.557102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f8b9f-2544-4966-8d94-bd9e9aecfa8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:45.557102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f8b9f-2544-4966-8d94-bd9e9aecfa8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:45.558366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f8b9f-2544-4966-8d94-bd9e9aecfa8f admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:45.569764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 953/1911] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:17:45.574187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f8b9f-2544-4966-8d94-bd9e9aecfa8f admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:45.584027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f8b9f-2544-4966-8d94-bd9e9aecfa8f admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:45.588784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f2337-f2bf-4df1-a54f-01b16d73afb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:45.589397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f2337-f2bf-4df1-a54f-01b16d73afb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.589673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f2337-f2bf-4df1-a54f-01b16d73afb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.589942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f2337-f2bf-4df1-a54f-01b16d73afb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.590555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f2337-f2bf-4df1-a54f-01b16d73afb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:45.592614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f2337-f2bf-4df1-a54f-01b16d73afb2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:45.619018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 959/1912] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:45 2022] POST /identity/v3/users => generated 292 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:45.629202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587009a1-f45c-4144-b3ef-1e4306eca9cf admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:45.630909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-587009a1-f45c-4144-b3ef-1e4306eca9cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:45.642485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587009a1-f45c-4144-b3ef-1e4306eca9cf admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:45.647205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d2f03-edf1-4c77-9542-75bb5f6d20d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:45.647814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d2f03-edf1-4c77-9542-75bb5f6d20d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.648090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d2f03-edf1-4c77-9542-75bb5f6d20d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.648362 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d2f03-edf1-4c77-9542-75bb5f6d20d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.649066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d2f03-edf1-4c77-9542-75bb5f6d20d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:45.650676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d2f03-edf1-4c77-9542-75bb5f6d20d3 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:45.662431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 960/1913] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:45.666269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d2f03-edf1-4c77-9542-75bb5f6d20d3 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:45.676885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d2f03-edf1-4c77-9542-75bb5f6d20d3 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:45.682785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc9249-ec4c-4a00-8741-c22c2d48b03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:45.683416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc9249-ec4c-4a00-8741-c22c2d48b03e admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.683697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc9249-ec4c-4a00-8741-c22c2d48b03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.683972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc9249-ec4c-4a00-8741-c22c2d48b03e admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.684613 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc9249-ec4c-4a00-8741-c22c2d48b03e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:45.686288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc9249-ec4c-4a00-8741-c22c2d48b03e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:45.697694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 961/1914] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:45.701943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc9249-ec4c-4a00-8741-c22c2d48b03e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:45.706901 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 954/1915] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:45 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:45.711037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f2337-f2bf-4df1-a54f-01b16d73afb2 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:45.721606 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc9249-ec4c-4a00-8741-c22c2d48b03e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:45.784018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa58fa-0f1c-415d-83e7-08157d19b72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:45.784607 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f2337-f2bf-4df1-a54f-01b16d73afb2 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:45.784712 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa58fa-0f1c-415d-83e7-08157d19b72b admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.784908 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa58fa-0f1c-415d-83e7-08157d19b72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.785212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa58fa-0f1c-415d-83e7-08157d19b72b admin admin] PATH_INFO: `/v3/projects/967392ca292846d2a3e728c908006273/users/a1586d4b73e44c9e8b67803765094707/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.794596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313646ed-ffb1-4e4f-89b3-5b6ba48ce303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:45.794945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313646ed-ffb1-4e4f-89b3-5b6ba48ce303 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.795180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313646ed-ffb1-4e4f-89b3-5b6ba48ce303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.795768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313646ed-ffb1-4e4f-89b3-5b6ba48ce303 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.796622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313646ed-ffb1-4e4f-89b3-5b6ba48ce303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:45.798952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313646ed-ffb1-4e4f-89b3-5b6ba48ce303 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:45.802363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa58fa-0f1c-415d-83e7-08157d19b72b admin admin] RBAC: Authorizing `identity:create_grant(project_id=967392ca292846d2a3e728c908006273, user_id=a1586d4b73e44c9e8b67803765094707, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:45.805796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa58fa-0f1c-415d-83e7-08157d19b72b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:45.815161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:45.909370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 962/1916] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:45 2022] PUT /identity/v3/projects/967392ca292846d2a3e728c908006273/users/a1586d4b73e44c9e8b67803765094707/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:45.922246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1818bb6-f6d8-471c-8212-21b0b4fbe0b3 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:45.922538 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1818bb6-f6d8-471c-8212-21b0b4fbe0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:45.922819 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1818bb6-f6d8-471c-8212-21b0b4fbe0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:45.947584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 955/1917] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:45 2022] POST /identity/v3/users => generated 294 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:45.953424 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b1818bb6-f6d8-471c-8212-21b0b4fbe0b3 None None] MFA Rules not processed for user `a1586d4b73e44c9e8b67803765094707`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:45.954656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b1818bb6-f6d8-471c-8212-21b0b4fbe0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:45.959521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313646ed-ffb1-4e4f-89b3-5b6ba48ce303 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:45.969946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313646ed-ffb1-4e4f-89b3-5b6ba48ce303 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.036888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 963/1918] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:46.039397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce509881-51ba-49f4-a835-c561427363a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.040113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1818bb6-f6d8-471c-8212-21b0b4fbe0b3 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.041088 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce509881-51ba-49f4-a835-c561427363a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.041506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce509881-51ba-49f4-a835-c561427363a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.042041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce509881-51ba-49f4-a835-c561427363a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.042812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce509881-51ba-49f4-a835-c561427363a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.044851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce509881-51ba-49f4-a835-c561427363a0 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.051770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1818bb6-f6d8-471c-8212-21b0b4fbe0b3 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.056110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 956/1919] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:46.060305 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce509881-51ba-49f4-a835-c561427363a0 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.060439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd377732-c790-4828-ba68-23630dc5cfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.060975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd377732-c790-4828-ba68-23630dc5cfc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.061277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd377732-c790-4828-ba68-23630dc5cfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.061661 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd377732-c790-4828-ba68-23630dc5cfc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.062449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd377732-c790-4828-ba68-23630dc5cfc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.064420 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd377732-c790-4828-ba68-23630dc5cfc9 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.073222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce509881-51ba-49f4-a835-c561427363a0 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.078307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628bf90-1e94-45c1-bf08-28cb5c80a8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.078928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628bf90-1e94-45c1-bf08-28cb5c80a8eb admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.079359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628bf90-1e94-45c1-bf08-28cb5c80a8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.079768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628bf90-1e94-45c1-bf08-28cb5c80a8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.080414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628bf90-1e94-45c1-bf08-28cb5c80a8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.082074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628bf90-1e94-45c1-bf08-28cb5c80a8eb admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.099394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 957/1920] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:46.101164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628bf90-1e94-45c1-bf08-28cb5c80a8eb admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.113777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628bf90-1e94-45c1-bf08-28cb5c80a8eb admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.118786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624adaa6-f38c-48f2-9dd8-dc06fa6ee7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.119442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624adaa6-f38c-48f2-9dd8-dc06fa6ee7ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.119722 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624adaa6-f38c-48f2-9dd8-dc06fa6ee7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.119992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624adaa6-f38c-48f2-9dd8-dc06fa6ee7ce admin admin] PATH_INFO: `/v3/projects/9b28d80d1f6c4b59903e6407126f1d46/users/01be315594664b0781e2df8ce6604666/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.138249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624adaa6-f38c-48f2-9dd8-dc06fa6ee7ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b28d80d1f6c4b59903e6407126f1d46, user_id=01be315594664b0781e2df8ce6604666, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.142973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624adaa6-f38c-48f2-9dd8-dc06fa6ee7ce admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.232934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 964/1921] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:46 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:46.236990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd377732-c790-4828-ba68-23630dc5cfc9 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.243610 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 958/1922] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:46 2022] PUT /identity/v3/projects/9b28d80d1f6c4b59903e6407126f1d46/users/01be315594664b0781e2df8ce6604666/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:46.253843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd3719-84a1-40fb-880f-ba4917723b8f None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.254135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd3719-84a1-40fb-880f-ba4917723b8f None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.254416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd3719-84a1-40fb-880f-ba4917723b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.285635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-8edd3719-84a1-40fb-880f-ba4917723b8f None None] MFA Rules not processed for user `01be315594664b0781e2df8ce6604666`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:46.287134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8edd3719-84a1-40fb-880f-ba4917723b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:46.301898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd377732-c790-4828-ba68-23630dc5cfc9 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.357661 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 959/1923] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:46.361257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e1051-d2df-46b1-9368-03d678af5c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.361257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e1051-d2df-46b1-9368-03d678af5c3b admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.361257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e1051-d2df-46b1-9368-03d678af5c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.361474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e1051-d2df-46b1-9368-03d678af5c3b admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.361961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e1051-d2df-46b1-9368-03d678af5c3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.362074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd3719-84a1-40fb-880f-ba4917723b8f None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.364045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e1051-d2df-46b1-9368-03d678af5c3b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.373666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd3719-84a1-40fb-880f-ba4917723b8f None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.378634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acb098-0032-4b72-81ba-fcabdf7d187f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.379236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acb098-0032-4b72-81ba-fcabdf7d187f admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.379517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acb098-0032-4b72-81ba-fcabdf7d187f admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.379790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acb098-0032-4b72-81ba-fcabdf7d187f admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.380422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acb098-0032-4b72-81ba-fcabdf7d187f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.382365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acb098-0032-4b72-81ba-fcabdf7d187f admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.478182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 965/1924] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:46 2022] POST /identity/v3/users => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:46.482465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e1051-d2df-46b1-9368-03d678af5c3b admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.492337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e1051-d2df-46b1-9368-03d678af5c3b admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.497932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a8500-3232-4651-8199-7060f166842b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.498529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a8500-3232-4651-8199-7060f166842b admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.498808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a8500-3232-4651-8199-7060f166842b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.499083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a8500-3232-4651-8199-7060f166842b admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.499820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a8500-3232-4651-8199-7060f166842b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.501441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a8500-3232-4651-8199-7060f166842b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.505183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 960/1925] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:46 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:46.506191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acb098-0032-4b72-81ba-fcabdf7d187f admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.512202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 966/1926] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:46.516323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a8500-3232-4651-8199-7060f166842b admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.526997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a8500-3232-4651-8199-7060f166842b admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.577171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acb098-0032-4b72-81ba-fcabdf7d187f admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.582765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c84084-8495-4cf4-9a40-446b8ac072a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.583935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c84084-8495-4cf4-9a40-446b8ac072a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.584216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c84084-8495-4cf4-9a40-446b8ac072a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.584488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c84084-8495-4cf4-9a40-446b8ac072a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.585156 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c84084-8495-4cf4-9a40-446b8ac072a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.587659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a62687-bfea-4af4-936b-a7921e74634b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.587799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c84084-8495-4cf4-9a40-446b8ac072a8 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.588534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a62687-bfea-4af4-936b-a7921e74634b admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.588806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a62687-bfea-4af4-936b-a7921e74634b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.589164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a62687-bfea-4af4-936b-a7921e74634b admin admin] PATH_INFO: `/v3/projects/70527565c1124e9bb8b4c290d2f2e455/users/3acc900cbf4c41d7a9e65468ad9cf68e/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.607076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a62687-bfea-4af4-936b-a7921e74634b admin admin] RBAC: Authorizing `identity:create_grant(project_id=70527565c1124e9bb8b4c290d2f2e455, user_id=3acc900cbf4c41d7a9e65468ad9cf68e, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.609984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a62687-bfea-4af4-936b-a7921e74634b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.694738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:46.751612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 961/1927] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:46 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:46.754800 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 967/1928] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:46 2022] PUT /identity/v3/projects/70527565c1124e9bb8b4c290d2f2e455/users/3acc900cbf4c41d7a9e65468ad9cf68e/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:46.755486 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c84084-8495-4cf4-9a40-446b8ac072a8 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.758681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a62687-bfea-4af4-936b-a7921e74634b admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.766461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c84084-8495-4cf4-9a40-446b8ac072a8 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.832798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6288f7-336a-4196-aae6-59dbd623068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.833099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a62687-bfea-4af4-936b-a7921e74634b admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.833619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6288f7-336a-4196-aae6-59dbd623068a admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.833857 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6288f7-336a-4196-aae6-59dbd623068a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.834141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6288f7-336a-4196-aae6-59dbd623068a admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.834793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6288f7-336a-4196-aae6-59dbd623068a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.836438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6288f7-336a-4196-aae6-59dbd623068a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.839204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf76bac-87b6-4c20-9af3-90e5e08477e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.839888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf76bac-87b6-4c20-9af3-90e5e08477e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.840234 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf76bac-87b6-4c20-9af3-90e5e08477e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.840574 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf76bac-87b6-4c20-9af3-90e5e08477e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.841318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf76bac-87b6-4c20-9af3-90e5e08477e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.843064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf76bac-87b6-4c20-9af3-90e5e08477e4 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.850751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 962/1929] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:46.851871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6288f7-336a-4196-aae6-59dbd623068a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.854664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 968/1930] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:46.858830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf76bac-87b6-4c20-9af3-90e5e08477e4 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:46.862668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6288f7-336a-4196-aae6-59dbd623068a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.867825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a4870-2248-4cb7-aedc-11c963bacc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.869368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a4870-2248-4cb7-aedc-11c963bacc8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.869650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a4870-2248-4cb7-aedc-11c963bacc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.872566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a4870-2248-4cb7-aedc-11c963bacc8c admin admin] PATH_INFO: `/v3/projects/15311911437349fb86a5f6f7a1cd5bf8/users/c87277bb91a24510926d4b71d0e1d5d3/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.872668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf76bac-87b6-4c20-9af3-90e5e08477e4 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:46.875612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1ac04-5fbd-448c-a68d-171e6b6ef5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:46.876266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1ac04-5fbd-448c-a68d-171e6b6ef5d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.876724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1ac04-5fbd-448c-a68d-171e6b6ef5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.878244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1ac04-5fbd-448c-a68d-171e6b6ef5d0 admin admin] PATH_INFO: `/v3/projects/70527565c1124e9bb8b4c290d2f2e455/users/3acc900cbf4c41d7a9e65468ad9cf68e/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.881108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1ac04-5fbd-448c-a68d-171e6b6ef5d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70527565c1124e9bb8b4c290d2f2e455, user_id=3acc900cbf4c41d7a9e65468ad9cf68e, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.883789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1ac04-5fbd-448c-a68d-171e6b6ef5d0 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.887462 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a4870-2248-4cb7-aedc-11c963bacc8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=15311911437349fb86a5f6f7a1cd5bf8, user_id=c87277bb91a24510926d4b71d0e1d5d3, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:46.890464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a4870-2248-4cb7-aedc-11c963bacc8c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:46.895840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 969/1931] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:46 2022] PUT /identity/v3/projects/70527565c1124e9bb8b4c290d2f2e455/users/3acc900cbf4c41d7a9e65468ad9cf68e/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:46.909179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afadc2e-574a-45ce-b0c8-2044ecfe33e5 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:46.909487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afadc2e-574a-45ce-b0c8-2044ecfe33e5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:46.910064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afadc2e-574a-45ce-b0c8-2044ecfe33e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:46.938669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0afadc2e-574a-45ce-b0c8-2044ecfe33e5 None None] MFA Rules not processed for user `3acc900cbf4c41d7a9e65468ad9cf68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:46.939934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0afadc2e-574a-45ce-b0c8-2044ecfe33e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:47.003106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 963/1932] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:46 2022] PUT /identity/v3/projects/15311911437349fb86a5f6f7a1cd5bf8/users/c87277bb91a24510926d4b71d0e1d5d3/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:47.007275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a4870-2248-4cb7-aedc-11c963bacc8c admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:47.011169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 970/1933] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:47.023078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afadc2e-574a-45ce-b0c8-2044ecfe33e5 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:47.024469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0afadc2e-574a-45ce-b0c8-2044ecfe33e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:47.079863 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a4870-2248-4cb7-aedc-11c963bacc8c admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:47.084752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244ea4e-4ecc-4a24-9227-d8eb282a0137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:47.085338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244ea4e-4ecc-4a24-9227-d8eb282a0137 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:47.085617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244ea4e-4ecc-4a24-9227-d8eb282a0137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:47.085870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afadc2e-574a-45ce-b0c8-2044ecfe33e5 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:47.085985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244ea4e-4ecc-4a24-9227-d8eb282a0137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:47.086598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244ea4e-4ecc-4a24-9227-d8eb282a0137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:47.088478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244ea4e-4ecc-4a24-9227-d8eb282a0137 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:47.098848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 964/1934] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:47.103619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244ea4e-4ecc-4a24-9227-d8eb282a0137 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:47.113561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244ea4e-4ecc-4a24-9227-d8eb282a0137 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:47.119882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267c287-4da6-4bb6-b692-cebc04be23f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:47.120550 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267c287-4da6-4bb6-b692-cebc04be23f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:47.120914 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267c287-4da6-4bb6-b692-cebc04be23f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:47.121290 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267c287-4da6-4bb6-b692-cebc04be23f8 admin admin] PATH_INFO: `/v3/projects/15311911437349fb86a5f6f7a1cd5bf8/users/c87277bb91a24510926d4b71d0e1d5d3/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:47.123910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267c287-4da6-4bb6-b692-cebc04be23f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15311911437349fb86a5f6f7a1cd5bf8, user_id=c87277bb91a24510926d4b71d0e1d5d3, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:47.126769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267c287-4da6-4bb6-b692-cebc04be23f8 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:47.142101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681be92-96f8-43e8-ae05-e3818c1b5d7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:47.142784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681be92-96f8-43e8-ae05-e3818c1b5d7c service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:47.143249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681be92-96f8-43e8-ae05-e3818c1b5d7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:47.143524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681be92-96f8-43e8-ae05-e3818c1b5d7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:47.145047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 965/1935] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:47 2022] PUT /identity/v3/projects/15311911437349fb86a5f6f7a1cd5bf8/users/c87277bb91a24510926d4b71d0e1d5d3/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:47.154826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d1e5f-ae89-4eb2-aad1-b5483e46e7a3 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:47.155258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d1e5f-ae89-4eb2-aad1-b5483e46e7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:47.155552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d1e5f-ae89-4eb2-aad1-b5483e46e7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:47.202088 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-740d1e5f-ae89-4eb2-aad1-b5483e46e7a3 None None] MFA Rules not processed for user `c87277bb91a24510926d4b71d0e1d5d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:47.202088 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-740d1e5f-ae89-4eb2-aad1-b5483e46e7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:47.241002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681be92-96f8-43e8-ae05-e3818c1b5d7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:47.243121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681be92-96f8-43e8-ae05-e3818c1b5d7c service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:47.249951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:47.255986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 971/1936] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:47.288713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 966/1937] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:47.299600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681be92-96f8-43e8-ae05-e3818c1b5d7c service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:47.309954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681be92-96f8-43e8-ae05-e3818c1b5d7c service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:47.374148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d1e5f-ae89-4eb2-aad1-b5483e46e7a3 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:47.378048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-740d1e5f-ae89-4eb2-aad1-b5483e46e7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:47.379620 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da45c56-5efe-470f-88c0-23e4fc90ba94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:47.380310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da45c56-5efe-470f-88c0-23e4fc90ba94 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:47.380707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da45c56-5efe-470f-88c0-23e4fc90ba94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:47.381019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da45c56-5efe-470f-88c0-23e4fc90ba94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:47.390598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d1e5f-ae89-4eb2-aad1-b5483e46e7a3 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:47.391654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da45c56-5efe-470f-88c0-23e4fc90ba94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:47.397462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da45c56-5efe-470f-88c0-23e4fc90ba94 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:47.397580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5500c-0334-41b5-8afe-2a08e2c6ec9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:47.397580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5500c-0334-41b5-8afe-2a08e2c6ec9b service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:47.397580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5500c-0334-41b5-8afe-2a08e2c6ec9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:47.397580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5500c-0334-41b5-8afe-2a08e2c6ec9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:47.404666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 972/1938] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:47.460743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5500c-0334-41b5-8afe-2a08e2c6ec9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:47.462920 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5500c-0334-41b5-8afe-2a08e2c6ec9b service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:47.472940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 967/1939] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:47.620954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:49.056498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:50.026799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da45c56-5efe-470f-88c0-23e4fc90ba94 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:50.107102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da45c56-5efe-470f-88c0-23e4fc90ba94 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:50.111830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:50.112485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:50.112762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:50.113082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] PATH_INFO: `/v3/users/01be315594664b0781e2df8ce6604666` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:50.114193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01be315594664b0781e2df8ce6604666)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:50.116103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:50.333296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01be315594664b0781e2df8ce6604666'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:50.346869 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01be315594664b0781e2df8ce6604666'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:50.346869 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01be315594664b0781e2df8ce6604666'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:50.356684 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01be315594664b0781e2df8ce6604666'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:50.516998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01be315594664b0781e2df8ce6604666'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:50.523726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 973/1940] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:50 2022] DELETE /identity/v3/users/01be315594664b0781e2df8ce6604666 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:50.528962 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf24e3-66da-48ef-97fc-ba4e7d0f2c11 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:50.529454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf24e3-66da-48ef-97fc-ba4e7d0f2c11 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:50.529553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf24e3-66da-48ef-97fc-ba4e7d0f2c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:50.550244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-06cf24e3-66da-48ef-97fc-ba4e7d0f2c11 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:50.551490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-06cf24e3-66da-48ef-97fc-ba4e7d0f2c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:50.613142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 968/1941] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:50.624632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:50.624792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:50.686148 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe952c-bd31-4bca-aaae-ae7305690a39 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:50.690200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ea531-fa50-4bfa-b8a5-9de3aac0deca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:50.690818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ea531-fa50-4bfa-b8a5-9de3aac0deca service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:50.691099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ea531-fa50-4bfa-b8a5-9de3aac0deca service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:50.691373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ea531-fa50-4bfa-b8a5-9de3aac0deca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:50.702160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ea531-fa50-4bfa-b8a5-9de3aac0deca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:50.703595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ea531-fa50-4bfa-b8a5-9de3aac0deca service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:50.715997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 974/1942] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:51.278437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf24e3-66da-48ef-97fc-ba4e7d0f2c11 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:51.284369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-06cf24e3-66da-48ef-97fc-ba4e7d0f2c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:51.301607 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf24e3-66da-48ef-97fc-ba4e7d0f2c11 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:51.307149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:51.307797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:51.308081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:51.308401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] PATH_INFO: `/v3/projects/9b28d80d1f6c4b59903e6407126f1d46` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:51.309680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b28d80d1f6c4b59903e6407126f1d46)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:51.311611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:51.343713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:51.468324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b28d80d1f6c4b59903e6407126f1d46', 'request_id': 'req-3a571328-16d7-41c9-8a0c-52854ea6a282'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:51.476391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b28d80d1f6c4b59903e6407126f1d46', 'request_id': 'req-3a571328-16d7-41c9-8a0c-52854ea6a282'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:51.485283 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b28d80d1f6c4b59903e6407126f1d46', 'request_id': 'req-3a571328-16d7-41c9-8a0c-52854ea6a282'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:51.590694 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] The token cache is being invalidate because project 9b28d80d1f6c4b59903e6407126f1d46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:51.594302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:51.594302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:51.594302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 969/1943] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:51 2022] DELETE /identity/v3/projects/9b28d80d1f6c4b59903e6407126f1d46 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:51.597528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ea531-fa50-4bfa-b8a5-9de3aac0deca service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:51.608782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ea531-fa50-4bfa-b8a5-9de3aac0deca service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:51.676073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112efed-98a6-4b02-82c4-ad7f369e2055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:51.676742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112efed-98a6-4b02-82c4-ad7f369e2055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:51.677094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112efed-98a6-4b02-82c4-ad7f369e2055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:51.677378 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112efed-98a6-4b02-82c4-ad7f369e2055 admin admin] PATH_INFO: `/v3/users/c87277bb91a24510926d4b71d0e1d5d3` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:51.678561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112efed-98a6-4b02-82c4-ad7f369e2055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87277bb91a24510926d4b71d0e1d5d3)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:51.680694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112efed-98a6-4b02-82c4-ad7f369e2055 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:51.842980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:51.844388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:51.858761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a571328-16d7-41c9-8a0c-52854ea6a282 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:51.863442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:51.864077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:51.864385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:51.864674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] PATH_INFO: `/v3/users/a1586d4b73e44c9e8b67803765094707` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:51.866430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1586d4b73e44c9e8b67803765094707)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:51.868559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:51.943634 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c112efed-98a6-4b02-82c4-ad7f369e2055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87277bb91a24510926d4b71d0e1d5d3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:51.944563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c112efed-98a6-4b02-82c4-ad7f369e2055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87277bb91a24510926d4b71d0e1d5d3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:51.954021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c112efed-98a6-4b02-82c4-ad7f369e2055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87277bb91a24510926d4b71d0e1d5d3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:51.968603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c112efed-98a6-4b02-82c4-ad7f369e2055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87277bb91a24510926d4b71d0e1d5d3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:52.143933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c112efed-98a6-4b02-82c4-ad7f369e2055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87277bb91a24510926d4b71d0e1d5d3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:52.152542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 975/1944] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:51 2022] DELETE /identity/v3/users/c87277bb91a24510926d4b71d0e1d5d3 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:52.213143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1586d4b73e44c9e8b67803765094707'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:52.214187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1586d4b73e44c9e8b67803765094707'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:52.224011 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1586d4b73e44c9e8b67803765094707'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:52.224822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1586d4b73e44c9e8b67803765094707'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:52.351136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1586d4b73e44c9e8b67803765094707'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:52.372914 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 970/1945] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:51 2022] DELETE /identity/v3/users/a1586d4b73e44c9e8b67803765094707 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:52.377823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca250c6c-0e12-4310-b84f-8cc32d95b021 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:52.378333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca250c6c-0e12-4310-b84f-8cc32d95b021 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:52.378454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca250c6c-0e12-4310-b84f-8cc32d95b021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:52.400268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ca250c6c-0e12-4310-b84f-8cc32d95b021 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:52.401725 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ca250c6c-0e12-4310-b84f-8cc32d95b021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:52.500453 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 976/1946] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:52.554896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:52.557795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:52.632173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84536256-c26d-4add-9a0b-c03c9c89fa71 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:52.641046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b0b9d-0c36-41f3-adbe-92dc123df247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:52.642061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0b9d-0c36-41f3-adbe-92dc123df247 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:52.642497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0b9d-0c36-41f3-adbe-92dc123df247 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:52.643091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0b9d-0c36-41f3-adbe-92dc123df247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:52.656827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b0b9d-0c36-41f3-adbe-92dc123df247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:52.659145 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b0b9d-0c36-41f3-adbe-92dc123df247 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:52.673386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 971/1947] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:52.826249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca250c6c-0e12-4310-b84f-8cc32d95b021 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:52.828137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ca250c6c-0e12-4310-b84f-8cc32d95b021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:52.864814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca250c6c-0e12-4310-b84f-8cc32d95b021 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:52.871985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:52.872780 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:52.873711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:52.873711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] PATH_INFO: `/v3/projects/15311911437349fb86a5f6f7a1cd5bf8` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:52.874742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=15311911437349fb86a5f6f7a1cd5bf8)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:52.876822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:52.894606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:53.014871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15311911437349fb86a5f6f7a1cd5bf8', 'request_id': 'req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:53.020482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15311911437349fb86a5f6f7a1cd5bf8', 'request_id': 'req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:53.098839 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15311911437349fb86a5f6f7a1cd5bf8', 'request_id': 'req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:53.117589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] The token cache is being invalidate because project 15311911437349fb86a5f6f7a1cd5bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:53.118602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:53.119510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-ad9a9a25-2c12-4bfb-b3ff-fa959a5f82fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:53.126677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 977/1948] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:52 2022] DELETE /identity/v3/projects/15311911437349fb86a5f6f7a1cd5bf8 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:53.128065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e21206-dc33-430b-a845-0cfabfbeb0c8 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:53.129901 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e21206-dc33-430b-a845-0cfabfbeb0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:53.130532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e21206-dc33-430b-a845-0cfabfbeb0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:53.158090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-33e21206-dc33-430b-a845-0cfabfbeb0c8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:53.159641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-33e21206-dc33-430b-a845-0cfabfbeb0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:53.254389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 972/1949] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:53.263186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0f20c-bc88-428b-ac16-bfc321f84752 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:53.264257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0f20c-bc88-428b-ac16-bfc321f84752 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:53.264257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0f20c-bc88-428b-ac16-bfc321f84752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:53.323895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-9ae0f20c-bc88-428b-ac16-bfc321f84752 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:53.335381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9ae0f20c-bc88-428b-ac16-bfc321f84752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:53.359785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 978/1950] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:53.361373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e21206-dc33-430b-a845-0cfabfbeb0c8 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:53.387030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e21206-dc33-430b-a845-0cfabfbeb0c8 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:53.401548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe67ff1-d256-44ca-92d7-75c4b1cd096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:53.402208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67ff1-d256-44ca-92d7-75c4b1cd096b admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:53.402704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67ff1-d256-44ca-92d7-75c4b1cd096b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:53.403423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67ff1-d256-44ca-92d7-75c4b1cd096b admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:53.404351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe67ff1-d256-44ca-92d7-75c4b1cd096b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:53.404351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe67ff1-d256-44ca-92d7-75c4b1cd096b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:53.409401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe67ff1-d256-44ca-92d7-75c4b1cd096b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:53.429971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 973/1951] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:53.434600 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0f20c-bc88-428b-ac16-bfc321f84752 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:53.445013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0f20c-bc88-428b-ac16-bfc321f84752 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:53.450467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160abdb-7669-426e-a8b4-60d84cab517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:53.451082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160abdb-7669-426e-a8b4-60d84cab517c admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:53.451360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160abdb-7669-426e-a8b4-60d84cab517c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:53.451819 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160abdb-7669-426e-a8b4-60d84cab517c admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:53.452255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160abdb-7669-426e-a8b4-60d84cab517c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:53.454340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160abdb-7669-426e-a8b4-60d84cab517c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:53.596373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 979/1952] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:53 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:53.600638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe67ff1-d256-44ca-92d7-75c4b1cd096b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:53.612584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe67ff1-d256-44ca-92d7-75c4b1cd096b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:53.697536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b45e5-a252-470f-abde-ca7e4e7e3a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:53.698223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b45e5-a252-470f-abde-ca7e4e7e3a3a admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:53.698428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b45e5-a252-470f-abde-ca7e4e7e3a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:53.698873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b45e5-a252-470f-abde-ca7e4e7e3a3a admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:53.699523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b45e5-a252-470f-abde-ca7e4e7e3a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:53.701503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b45e5-a252-470f-abde-ca7e4e7e3a3a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:53.961314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160abdb-7669-426e-a8b4-60d84cab517c admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:53.964294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f160abdb-7669-426e-a8b4-60d84cab517c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:53.990050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160abdb-7669-426e-a8b4-60d84cab517c admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:53.995938 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539da524-a317-4b98-a25a-68a15d8b32e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:53.996505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539da524-a317-4b98-a25a-68a15d8b32e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:53.996787 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539da524-a317-4b98-a25a-68a15d8b32e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:53.997111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539da524-a317-4b98-a25a-68a15d8b32e0 admin admin] PATH_INFO: `/v3/users/07506462884949f1839d1e9dcc86e513` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:53.998242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539da524-a317-4b98-a25a-68a15d8b32e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07506462884949f1839d1e9dcc86e513)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:54.000210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539da524-a317-4b98-a25a-68a15d8b32e0 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:54.115496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 974/1953] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:53 2022] POST /identity/v3/users => generated 291 bytes in 515 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:54.119718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b45e5-a252-470f-abde-ca7e4e7e3a3a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:54.129885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b45e5-a252-470f-abde-ca7e4e7e3a3a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:54.135225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a025a7f-e42b-4f89-9580-4cec8be2238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:54.136010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a025a7f-e42b-4f89-9580-4cec8be2238b admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:54.136455 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a025a7f-e42b-4f89-9580-4cec8be2238b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:54.136730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a025a7f-e42b-4f89-9580-4cec8be2238b admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:54.137478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a025a7f-e42b-4f89-9580-4cec8be2238b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:54.139362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a025a7f-e42b-4f89-9580-4cec8be2238b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:54.152478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 975/1954] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:54.158391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a025a7f-e42b-4f89-9580-4cec8be2238b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:54.173579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a025a7f-e42b-4f89-9580-4cec8be2238b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:54.179385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70a21d-211d-4168-826c-3c17cf824c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:54.180668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70a21d-211d-4168-826c-3c17cf824c3d admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:54.181190 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70a21d-211d-4168-826c-3c17cf824c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:54.181761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70a21d-211d-4168-826c-3c17cf824c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:54.182891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70a21d-211d-4168-826c-3c17cf824c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:54.184691 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70a21d-211d-4168-826c-3c17cf824c3d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:54.198794 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 976/1955] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:54.208147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70a21d-211d-4168-826c-3c17cf824c3d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:54.220223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70a21d-211d-4168-826c-3c17cf824c3d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:54.226366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866c069-b721-4c19-9905-81b352ffd99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:54.227869 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866c069-b721-4c19-9905-81b352ffd99a admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:54.228162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866c069-b721-4c19-9905-81b352ffd99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:54.228574 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866c069-b721-4c19-9905-81b352ffd99a admin admin] PATH_INFO: `/v3/projects/959f43a380d14f0db061b5bd6da919f7/users/30ac7259a76b427aa8db0137b84be8f3/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:54.257199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866c069-b721-4c19-9905-81b352ffd99a admin admin] RBAC: Authorizing `identity:create_grant(project_id=959f43a380d14f0db061b5bd6da919f7, user_id=30ac7259a76b427aa8db0137b84be8f3, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:54.257199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866c069-b721-4c19-9905-81b352ffd99a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:54.316554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-539da524-a317-4b98-a25a-68a15d8b32e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07506462884949f1839d1e9dcc86e513'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:54.316897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-539da524-a317-4b98-a25a-68a15d8b32e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07506462884949f1839d1e9dcc86e513'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:54.328203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-539da524-a317-4b98-a25a-68a15d8b32e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07506462884949f1839d1e9dcc86e513'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:54.345917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-539da524-a317-4b98-a25a-68a15d8b32e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07506462884949f1839d1e9dcc86e513'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:54.482639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-539da524-a317-4b98-a25a-68a15d8b32e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07506462884949f1839d1e9dcc86e513'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:54.489292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 977/1956] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:54 2022] PUT /identity/v3/projects/959f43a380d14f0db061b5bd6da919f7/users/30ac7259a76b427aa8db0137b84be8f3/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:54.490302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866c069-b721-4c19-9905-81b352ffd99a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:54.493071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3866c069-b721-4c19-9905-81b352ffd99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:54.494861 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 980/1957] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:53 2022] DELETE /identity/v3/users/07506462884949f1839d1e9dcc86e513 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:54.509717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e30202-996e-4aaf-9bc0-17787d6823fc None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:54.509913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e30202-996e-4aaf-9bc0-17787d6823fc None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:54.510189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e30202-996e-4aaf-9bc0-17787d6823fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:54.571912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-62e30202-996e-4aaf-9bc0-17787d6823fc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:54.573495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-62e30202-996e-4aaf-9bc0-17787d6823fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:54.590679 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866c069-b721-4c19-9905-81b352ffd99a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:54.703315 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:54.704117 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:54.704513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:54.704876 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] PATH_INFO: `/v3/projects/967392ca292846d2a3e728c908006273` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:54.706816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=967392ca292846d2a3e728c908006273)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:54.709611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:54.720573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 981/1958] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:54.722653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90337dbf-fc3c-44ad-96f7-2e02a57e5e24 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:54.722653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90337dbf-fc3c-44ad-96f7-2e02a57e5e24 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:54.722898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90337dbf-fc3c-44ad-96f7-2e02a57e5e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:54.769120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-90337dbf-fc3c-44ad-96f7-2e02a57e5e24 None None] MFA Rules not processed for user `30ac7259a76b427aa8db0137b84be8f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:54.770691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-90337dbf-fc3c-44ad-96f7-2e02a57e5e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:54.800666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:54.862484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 982/1959] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:54.863231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90337dbf-fc3c-44ad-96f7-2e02a57e5e24 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:54.864789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-90337dbf-fc3c-44ad-96f7-2e02a57e5e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:54.943617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '967392ca292846d2a3e728c908006273', 'request_id': 'req-6be39113-d094-4082-929b-41fe2c5218b2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:54.960334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90337dbf-fc3c-44ad-96f7-2e02a57e5e24 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:54.966056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '967392ca292846d2a3e728c908006273', 'request_id': 'req-6be39113-d094-4082-929b-41fe2c5218b2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:54.986881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '967392ca292846d2a3e728c908006273', 'request_id': 'req-6be39113-d094-4082-929b-41fe2c5218b2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:55.030195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2823212-522a-4096-8d6a-968ab0f17b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:55.030830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2823212-522a-4096-8d6a-968ab0f17b1e service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:55.031115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2823212-522a-4096-8d6a-968ab0f17b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:55.031534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2823212-522a-4096-8d6a-968ab0f17b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:55.105364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2823212-522a-4096-8d6a-968ab0f17b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:55.107081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2823212-522a-4096-8d6a-968ab0f17b1e service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:55.111040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] The token cache is being invalidate because project 967392ca292846d2a3e728c908006273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:55.111340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:55.112175 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:55.118879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 978/1960] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:54 2022] DELETE /identity/v3/projects/967392ca292846d2a3e728c908006273 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:55.120598 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 983/1961] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:55.121204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:55.121509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2823212-522a-4096-8d6a-968ab0f17b1e service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:55.123563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:55.135335 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2823212-522a-4096-8d6a-968ab0f17b1e service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:55.140700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:55.145833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:55.145833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:55.145833 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] PATH_INFO: `/v3/users/3acc900cbf4c41d7a9e65468ad9cf68e` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:55.146097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be39113-d094-4082-929b-41fe2c5218b2 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:55.146188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3acc900cbf4c41d7a9e65468ad9cf68e)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:55.146188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:55.147947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc13c45-f829-4912-b6b8-766dcad6d188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:55.148720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc13c45-f829-4912-b6b8-766dcad6d188 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:55.149729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc13c45-f829-4912-b6b8-766dcad6d188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:55.150145 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc13c45-f829-4912-b6b8-766dcad6d188 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:55.151195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc13c45-f829-4912-b6b8-766dcad6d188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:55.153315 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc13c45-f829-4912-b6b8-766dcad6d188 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:55.256070 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 979/1962] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:55 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:55.261694 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc13c45-f829-4912-b6b8-766dcad6d188 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:55.347226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc13c45-f829-4912-b6b8-766dcad6d188 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:55.353048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033910a6-8357-403a-b20f-25620a065b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:55.353865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033910a6-8357-403a-b20f-25620a065b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:55.354108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033910a6-8357-403a-b20f-25620a065b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:55.354286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033910a6-8357-403a-b20f-25620a065b85 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:55.355061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033910a6-8357-403a-b20f-25620a065b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:55.357060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033910a6-8357-403a-b20f-25620a065b85 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:55.458158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3acc900cbf4c41d7a9e65468ad9cf68e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:55.458158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3acc900cbf4c41d7a9e65468ad9cf68e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:55.471883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3acc900cbf4c41d7a9e65468ad9cf68e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:55.486588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 980/1963] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:55 2022] POST /identity/v3/users => generated 296 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:55.487521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033910a6-8357-403a-b20f-25620a065b85 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:55.501032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3acc900cbf4c41d7a9e65468ad9cf68e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:55.504270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033910a6-8357-403a-b20f-25620a065b85 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:55.510904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43a739-330e-421d-b528-1ce09fd158f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:55.511766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43a739-330e-421d-b528-1ce09fd158f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:55.512320 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43a739-330e-421d-b528-1ce09fd158f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:55.512599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43a739-330e-421d-b528-1ce09fd158f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:55.514300 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43a739-330e-421d-b528-1ce09fd158f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:55.516405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43a739-330e-421d-b528-1ce09fd158f8 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:55.549496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 981/1964] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:55.553668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43a739-330e-421d-b528-1ce09fd158f8 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:55.568879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43a739-330e-421d-b528-1ce09fd158f8 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:55.576189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d093911-a770-4b66-8709-0e96a7a532c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:55.576828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d093911-a770-4b66-8709-0e96a7a532c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:55.577265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d093911-a770-4b66-8709-0e96a7a532c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:55.577663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d093911-a770-4b66-8709-0e96a7a532c9 admin admin] PATH_INFO: `/v3/projects/4e3b431223ce4441b07437c7b7ecb34c/users/0efb0b109c0346ed903466bfc5c3160b/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:55.604556 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3acc900cbf4c41d7a9e65468ad9cf68e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:55.615027 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 984/1965] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:55 2022] DELETE /identity/v3/users/3acc900cbf4c41d7a9e65468ad9cf68e => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:55.616469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d093911-a770-4b66-8709-0e96a7a532c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e3b431223ce4441b07437c7b7ecb34c, user_id=0efb0b109c0346ed903466bfc5c3160b, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:55.620298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d093911-a770-4b66-8709-0e96a7a532c9 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:55.663728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:55.719223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 982/1966] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:55 2022] PUT /identity/v3/projects/4e3b431223ce4441b07437c7b7ecb34c/users/0efb0b109c0346ed903466bfc5c3160b/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:55.723452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:55.724964 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:55.778622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d093911-a770-4b66-8709-0e96a7a532c9 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:55.781165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1d093911-a770-4b66-8709-0e96a7a532c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:55.875616 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44b1db-27a1-4e57-93a7-084e158ace48 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:55.947312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78641-5bdb-447f-ad2b-adf576577f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:55.948000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78641-5bdb-447f-ad2b-adf576577f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:55.948285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78641-5bdb-447f-ad2b-adf576577f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:55.948560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78641-5bdb-447f-ad2b-adf576577f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:55.949329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc78641-5bdb-447f-ad2b-adf576577f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:55.951117 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d093911-a770-4b66-8709-0e96a7a532c9 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:55.951739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc78641-5bdb-447f-ad2b-adf576577f1f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:55.968186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 985/1967] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:55.968679 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:55.969245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78641-5bdb-447f-ad2b-adf576577f1f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:55.969583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:55.969865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:55.970138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] PATH_INFO: `/v3/projects/d55b789674d5481f97fd3cb33fdc3f40` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:55.972521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d55b789674d5481f97fd3cb33fdc3f40)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:55.977286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:55.983195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78641-5bdb-447f-ad2b-adf576577f1f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:56.006809 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3b29f-976f-476c-b104-8c1eab48424d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:56.008993 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3b29f-976f-476c-b104-8c1eab48424d admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:56.009438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3b29f-976f-476c-b104-8c1eab48424d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:56.009895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3b29f-976f-476c-b104-8c1eab48424d admin admin] PATH_INFO: `/v3/projects/4e3b431223ce4441b07437c7b7ecb34c/users/0efb0b109c0346ed903466bfc5c3160b/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:56.014353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3b29f-976f-476c-b104-8c1eab48424d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e3b431223ce4441b07437c7b7ecb34c, user_id=0efb0b109c0346ed903466bfc5c3160b, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:56.021647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3b29f-976f-476c-b104-8c1eab48424d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:56.051101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 986/1968] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:55 2022] PUT /identity/v3/projects/4e3b431223ce4441b07437c7b7ecb34c/users/0efb0b109c0346ed903466bfc5c3160b/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:56.065995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6d1c0-0865-444e-b492-ad52f80e467e None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:56.067036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6d1c0-0865-444e-b492-ad52f80e467e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:56.067036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6d1c0-0865-444e-b492-ad52f80e467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:56.161304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-2ff6d1c0-0865-444e-b492-ad52f80e467e None None] MFA Rules not processed for user `0efb0b109c0346ed903466bfc5c3160b`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:56.162764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2ff6d1c0-0865-444e-b492-ad52f80e467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:56.176311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd55b789674d5481f97fd3cb33fdc3f40', 'request_id': 'req-a1caafc2-41dd-4fdd-aa02-2b872e187838'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:56.191221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd55b789674d5481f97fd3cb33fdc3f40', 'request_id': 'req-a1caafc2-41dd-4fdd-aa02-2b872e187838'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:56.206223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd55b789674d5481f97fd3cb33fdc3f40', 'request_id': 'req-a1caafc2-41dd-4fdd-aa02-2b872e187838'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:56.287086 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 987/1969] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:56.295689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6d1c0-0865-444e-b492-ad52f80e467e None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:56.298456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] The token cache is being invalidate because project d55b789674d5481f97fd3cb33fdc3f40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:56.298760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:56.299439 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:56.301277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 983/1970] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:55 2022] DELETE /identity/v3/projects/d55b789674d5481f97fd3cb33fdc3f40 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:56.302297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2ff6d1c0-0865-444e-b492-ad52f80e467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:56.459354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6d1c0-0865-444e-b492-ad52f80e467e None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:56.464768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663fb20-fcd1-45fe-9120-082801fd42a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:56.465452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663fb20-fcd1-45fe-9120-082801fd42a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:56.465794 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663fb20-fcd1-45fe-9120-082801fd42a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:56.466080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663fb20-fcd1-45fe-9120-082801fd42a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:56.468190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7663fb20-fcd1-45fe-9120-082801fd42a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:56.483929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663fb20-fcd1-45fe-9120-082801fd42a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:56.485538 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663fb20-fcd1-45fe-9120-082801fd42a0 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:56.502493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 988/1971] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:56.711133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:56.713030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:56.727453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caafc2-41dd-4fdd-aa02-2b872e187838 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:56.731742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2c99e-d9dc-44a2-9efe-307750ef03aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:56.732519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2c99e-d9dc-44a2-9efe-307750ef03aa service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:56.732802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2c99e-d9dc-44a2-9efe-307750ef03aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:56.733339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2c99e-d9dc-44a2-9efe-307750ef03aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:56.735199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7ca2c99e-d9dc-44a2-9efe-307750ef03aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:56.835443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2c99e-d9dc-44a2-9efe-307750ef03aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:56.837732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2c99e-d9dc-44a2-9efe-307750ef03aa service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:56.862652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 984/1972] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:56.871234 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663fb20-fcd1-45fe-9120-082801fd42a0 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:56.874078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7663fb20-fcd1-45fe-9120-082801fd42a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:56.949796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:56.997826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663fb20-fcd1-45fe-9120-082801fd42a0 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:57.002550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:57.003214 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:57.003497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:57.003775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] PATH_INFO: `/v3/projects/70527565c1124e9bb8b4c290d2f2e455` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:57.005143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=70527565c1124e9bb8b4c290d2f2e455)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:57.007084 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:57.141440 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70527565c1124e9bb8b4c290d2f2e455', 'request_id': 'req-3f7d14ea-92a4-4402-9054-effd636bfe5f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:57.147342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70527565c1124e9bb8b4c290d2f2e455', 'request_id': 'req-3f7d14ea-92a4-4402-9054-effd636bfe5f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:57.225453 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70527565c1124e9bb8b4c290d2f2e455', 'request_id': 'req-3f7d14ea-92a4-4402-9054-effd636bfe5f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:57.237982 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] The token cache is being invalidate because project 70527565c1124e9bb8b4c290d2f2e455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:57.238291 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:57.238949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3f7d14ea-92a4-4402-9054-effd636bfe5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:57.240507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 989/1973] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:56 2022] DELETE /identity/v3/projects/70527565c1124e9bb8b4c290d2f2e455 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:57.247901 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2704e3-f09b-4f7f-be22-eb542d8709c2 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:57.248229 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2704e3-f09b-4f7f-be22-eb542d8709c2 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:57.248504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2704e3-f09b-4f7f-be22-eb542d8709c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:57.271024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-fd2704e3-f09b-4f7f-be22-eb542d8709c2 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:57.272249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-fd2704e3-f09b-4f7f-be22-eb542d8709c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:57.336551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 985/1974] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:57.346511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce106c-e3be-49bc-9fd9-69ded60c9ff2 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:57.346511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce106c-e3be-49bc-9fd9-69ded60c9ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:57.346511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce106c-e3be-49bc-9fd9-69ded60c9ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:57.368906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-bcce106c-e3be-49bc-9fd9-69ded60c9ff2 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:57.370157 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-bcce106c-e3be-49bc-9fd9-69ded60c9ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:57.381940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 990/1975] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:57.383264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2704e3-f09b-4f7f-be22-eb542d8709c2 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:57.394438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2704e3-f09b-4f7f-be22-eb542d8709c2 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:57.399859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7888d-819d-433a-961e-cc5c9544d075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:57.400468 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7888d-819d-433a-961e-cc5c9544d075 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:57.400752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7888d-819d-433a-961e-cc5c9544d075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:57.401066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7888d-819d-433a-961e-cc5c9544d075 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:57.401717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7888d-819d-433a-961e-cc5c9544d075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:57.402023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7888d-819d-433a-961e-cc5c9544d075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:57.403618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7888d-819d-433a-961e-cc5c9544d075 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:57.420033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 986/1976] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:57.422327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce106c-e3be-49bc-9fd9-69ded60c9ff2 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:57.433063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce106c-e3be-49bc-9fd9-69ded60c9ff2 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:57.438031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe63972-4db8-4071-a15b-cc5e20a1186c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:57.438632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe63972-4db8-4071-a15b-cc5e20a1186c admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:57.443502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe63972-4db8-4071-a15b-cc5e20a1186c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:57.443502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe63972-4db8-4071-a15b-cc5e20a1186c admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:57.443502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe63972-4db8-4071-a15b-cc5e20a1186c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:57.443502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe63972-4db8-4071-a15b-cc5e20a1186c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:57.458905 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7888d-819d-433a-961e-cc5c9544d075 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:57.461899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9ef7888d-819d-433a-961e-cc5c9544d075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:57.475379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7888d-819d-433a-961e-cc5c9544d075 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:57.480847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d126f-242b-48b6-a717-018a1fc3a55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:57.481518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d126f-242b-48b6-a717-018a1fc3a55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:57.481799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d126f-242b-48b6-a717-018a1fc3a55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:57.482073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d126f-242b-48b6-a717-018a1fc3a55f admin admin] PATH_INFO: `/v3/users/30ac7259a76b427aa8db0137b84be8f3` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:57.483205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d126f-242b-48b6-a717-018a1fc3a55f admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ac7259a76b427aa8db0137b84be8f3)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:57.485153 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d126f-242b-48b6-a717-018a1fc3a55f admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:57.552190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 991/1977] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:57 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:57.557921 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe63972-4db8-4071-a15b-cc5e20a1186c admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:57.588953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:57.624154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe63972-4db8-4071-a15b-cc5e20a1186c admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:57.628851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b2e2d-e8f4-43c0-aad9-c39279549edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:57.629566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b2e2d-e8f4-43c0-aad9-c39279549edd admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:57.629847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b2e2d-e8f4-43c0-aad9-c39279549edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:57.630124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b2e2d-e8f4-43c0-aad9-c39279549edd admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:57.630748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b2e2d-e8f4-43c0-aad9-c39279549edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:57.634566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b2e2d-e8f4-43c0-aad9-c39279549edd admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:57.682828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:57.781552 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 992/1978] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:57 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:57.784996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b2e2d-e8f4-43c0-aad9-c39279549edd admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:57.795426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b2e2d-e8f4-43c0-aad9-c39279549edd admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:57.800040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0bc1c-6ee7-4dad-8f5b-bdd5b5f26d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:57.800671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0bc1c-6ee7-4dad-8f5b-bdd5b5f26d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:57.800972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0bc1c-6ee7-4dad-8f5b-bdd5b5f26d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:57.801253 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0bc1c-6ee7-4dad-8f5b-bdd5b5f26d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:57.802212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0bc1c-6ee7-4dad-8f5b-bdd5b5f26d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:57.803629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0bc1c-6ee7-4dad-8f5b-bdd5b5f26d81 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:57.817564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 993/1979] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:57.818933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0bc1c-6ee7-4dad-8f5b-bdd5b5f26d81 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:57.830167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0bc1c-6ee7-4dad-8f5b-bdd5b5f26d81 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:57.838320 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7b6d126f-242b-48b6-a717-018a1fc3a55f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ac7259a76b427aa8db0137b84be8f3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:57.839142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7b6d126f-242b-48b6-a717-018a1fc3a55f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ac7259a76b427aa8db0137b84be8f3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:57.843562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abbd75-357c-44f3-91c7-7519adcd8665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:57.844200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abbd75-357c-44f3-91c7-7519adcd8665 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:57.844487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abbd75-357c-44f3-91c7-7519adcd8665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:57.844769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abbd75-357c-44f3-91c7-7519adcd8665 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:57.845454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abbd75-357c-44f3-91c7-7519adcd8665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:57.846551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7b6d126f-242b-48b6-a717-018a1fc3a55f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ac7259a76b427aa8db0137b84be8f3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:57.847072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abbd75-357c-44f3-91c7-7519adcd8665 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:57.847546 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7b6d126f-242b-48b6-a717-018a1fc3a55f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ac7259a76b427aa8db0137b84be8f3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:57.862063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 994/1980] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:57.863105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abbd75-357c-44f3-91c7-7519adcd8665 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:57.864842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-39abbd75-357c-44f3-91c7-7519adcd8665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:57.879421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abbd75-357c-44f3-91c7-7519adcd8665 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:57.886287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaad50-577e-4902-be54-960a3e9d77b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:57.887106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaad50-577e-4902-be54-960a3e9d77b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:57.887463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaad50-577e-4902-be54-960a3e9d77b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:57.887805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaad50-577e-4902-be54-960a3e9d77b8 admin admin] PATH_INFO: `/v3/projects/a23b541a4f0b45dd876920f17860226a/users/bedc32c7be70426ea5654fe710f1b775/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:57.910388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaad50-577e-4902-be54-960a3e9d77b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23b541a4f0b45dd876920f17860226a, user_id=bedc32c7be70426ea5654fe710f1b775, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:57.913973 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaad50-577e-4902-be54-960a3e9d77b8 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:57.962380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7b6d126f-242b-48b6-a717-018a1fc3a55f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ac7259a76b427aa8db0137b84be8f3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:57.997871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 987/1981] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:57 2022] DELETE /identity/v3/users/30ac7259a76b427aa8db0137b84be8f3 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:58.004727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 995/1982] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:57 2022] PUT /identity/v3/projects/a23b541a4f0b45dd876920f17860226a/users/bedc32c7be70426ea5654fe710f1b775/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:58.007328 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ae8a4-6dc0-4299-a449-122be64effb0 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:58.007697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ae8a4-6dc0-4299-a449-122be64effb0 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:58.008108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ae8a4-6dc0-4299-a449-122be64effb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:58.020109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6054b-a769-47b2-bfae-7ce67497bc01 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:58.020481 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6054b-a769-47b2-bfae-7ce67497bc01 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:58.020768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6054b-a769-47b2-bfae-7ce67497bc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:58.030840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-7d1ae8a4-6dc0-4299-a449-122be64effb0 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:58.032348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7d1ae8a4-6dc0-4299-a449-122be64effb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:58.056704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-c1c6054b-a769-47b2-bfae-7ce67497bc01 None None] MFA Rules not processed for user `bedc32c7be70426ea5654fe710f1b775`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:58.059530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c1c6054b-a769-47b2-bfae-7ce67497bc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:58.107856 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 988/1983] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:58.120187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ae8a4-6dc0-4299-a449-122be64effb0 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:58.122208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7d1ae8a4-6dc0-4299-a449-122be64effb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:58.153944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 996/1984] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:58.166940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6054b-a769-47b2-bfae-7ce67497bc01 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:58.168827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c1c6054b-a769-47b2-bfae-7ce67497bc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:58.196426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ae8a4-6dc0-4299-a449-122be64effb0 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:58.200346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733b61e-b1a2-43ff-a0fc-b46c7f7bc121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:58.206831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733b61e-b1a2-43ff-a0fc-b46c7f7bc121 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:58.206831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733b61e-b1a2-43ff-a0fc-b46c7f7bc121 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:58.206831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733b61e-b1a2-43ff-a0fc-b46c7f7bc121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:58.211212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733b61e-b1a2-43ff-a0fc-b46c7f7bc121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:58.212901 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733b61e-b1a2-43ff-a0fc-b46c7f7bc121 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:58.237584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 989/1985] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:58.244997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6054b-a769-47b2-bfae-7ce67497bc01 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:58.249379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b21b0-b229-462b-a5cc-9de7ca02c4f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:58.250548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b21b0-b229-462b-a5cc-9de7ca02c4f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:58.250548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b21b0-b229-462b-a5cc-9de7ca02c4f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:58.250548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b21b0-b229-462b-a5cc-9de7ca02c4f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:58.266314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b21b0-b229-462b-a5cc-9de7ca02c4f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:58.267115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b21b0-b229-462b-a5cc-9de7ca02c4f0 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:58.281430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 997/1986] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:58.308514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733b61e-b1a2-43ff-a0fc-b46c7f7bc121 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:58.319126 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733b61e-b1a2-43ff-a0fc-b46c7f7bc121 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:58.325904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:58.326530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:58.326815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:58.327102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] PATH_INFO: `/v3/users/bedc32c7be70426ea5654fe710f1b775` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:58.328806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bedc32c7be70426ea5654fe710f1b775)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:58.330798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:58.617637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bedc32c7be70426ea5654fe710f1b775'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:58.618559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bedc32c7be70426ea5654fe710f1b775'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:58.629934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bedc32c7be70426ea5654fe710f1b775'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:58.630807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bedc32c7be70426ea5654fe710f1b775'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:58.779269 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bedc32c7be70426ea5654fe710f1b775'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:58.804507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 990/1987] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:58 2022] DELETE /identity/v3/users/bedc32c7be70426ea5654fe710f1b775 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:58.809749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac0e45-0eff-4275-8b1d-cf0df5340fef None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:58.810230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac0e45-0eff-4275-8b1d-cf0df5340fef None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:58.810589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac0e45-0eff-4275-8b1d-cf0df5340fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:58.830886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-4cac0e45-0eff-4275-8b1d-cf0df5340fef None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:58.832132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4cac0e45-0eff-4275-8b1d-cf0df5340fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:58.850984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:58.853173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:58.907180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 998/1988] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:58.917700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac0e45-0eff-4275-8b1d-cf0df5340fef None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:58.925264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4cac0e45-0eff-4275-8b1d-cf0df5340fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:58.954370 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6f723-21c4-4fc1-8e3a-26b57e0ab3f5 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:58.960238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:58.960887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:58.962117 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:58.962329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] PATH_INFO: `/v3/projects/959f43a380d14f0db061b5bd6da919f7` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:58.963898 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] RBAC: Authorizing `identity:delete_project(project_id=959f43a380d14f0db061b5bd6da919f7)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:58.965522 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:58.993723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac0e45-0eff-4275-8b1d-cf0df5340fef None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:58.998938 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b1a44-8452-4cd8-83a1-9123147ef80b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:59.003167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b1a44-8452-4cd8-83a1-9123147ef80b service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:59.003167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b1a44-8452-4cd8-83a1-9123147ef80b service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:59.003167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b1a44-8452-4cd8-83a1-9123147ef80b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:59.011822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b1a44-8452-4cd8-83a1-9123147ef80b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:59.013393 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b1a44-8452-4cd8-83a1-9123147ef80b service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:59.027784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 999/1989] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:59.107862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '959f43a380d14f0db061b5bd6da919f7', 'request_id': 'req-880b6442-ff61-4cc0-ac3a-15a67b41e08a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.113284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '959f43a380d14f0db061b5bd6da919f7', 'request_id': 'req-880b6442-ff61-4cc0-ac3a-15a67b41e08a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.121448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '959f43a380d14f0db061b5bd6da919f7', 'request_id': 'req-880b6442-ff61-4cc0-ac3a-15a67b41e08a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.253510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] The token cache is being invalidate because project 959f43a380d14f0db061b5bd6da919f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:59.253704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.254362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.256031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 991/1990] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:58 2022] DELETE /identity/v3/projects/959f43a380d14f0db061b5bd6da919f7 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:59.260887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b1a44-8452-4cd8-83a1-9123147ef80b service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:59.271777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b1a44-8452-4cd8-83a1-9123147ef80b service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:59.343026 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be986c67-7cb7-4a0c-a18e-57e4bbb81469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:59.343676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be986c67-7cb7-4a0c-a18e-57e4bbb81469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:59.343969 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be986c67-7cb7-4a0c-a18e-57e4bbb81469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:59.344268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be986c67-7cb7-4a0c-a18e-57e4bbb81469 admin admin] PATH_INFO: `/v3/users/0efb0b109c0346ed903466bfc5c3160b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:59.345413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be986c67-7cb7-4a0c-a18e-57e4bbb81469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0efb0b109c0346ed903466bfc5c3160b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:59.347331 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be986c67-7cb7-4a0c-a18e-57e4bbb81469 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:59.567756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:59.569439 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:59.585174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b6442-ff61-4cc0-ac3a-15a67b41e08a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:59.590188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:59.590899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:59.591184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:59.591621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] PATH_INFO: `/v3/projects/a23b541a4f0b45dd876920f17860226a` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:59.592792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a23b541a4f0b45dd876920f17860226a)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:59.594902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:59.596210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-be986c67-7cb7-4a0c-a18e-57e4bbb81469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0efb0b109c0346ed903466bfc5c3160b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.597778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-be986c67-7cb7-4a0c-a18e-57e4bbb81469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0efb0b109c0346ed903466bfc5c3160b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.607659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-be986c67-7cb7-4a0c-a18e-57e4bbb81469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0efb0b109c0346ed903466bfc5c3160b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.621436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-be986c67-7cb7-4a0c-a18e-57e4bbb81469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0efb0b109c0346ed903466bfc5c3160b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.723037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a23b541a4f0b45dd876920f17860226a', 'request_id': 'req-8cce7462-a32d-4db6-ace9-c592e9812717'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.729464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a23b541a4f0b45dd876920f17860226a', 'request_id': 'req-8cce7462-a32d-4db6-ace9-c592e9812717'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.733876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-be986c67-7cb7-4a0c-a18e-57e4bbb81469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0efb0b109c0346ed903466bfc5c3160b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.736222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a23b541a4f0b45dd876920f17860226a', 'request_id': 'req-8cce7462-a32d-4db6-ace9-c592e9812717'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.740699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1000/1991] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:59 2022] DELETE /identity/v3/users/0efb0b109c0346ed903466bfc5c3160b => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:59.906752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] The token cache is being invalidate because project a23b541a4f0b45dd876920f17860226a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:59.907252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.907762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-8cce7462-a32d-4db6-ace9-c592e9812717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:59.911854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 992/1992] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:59 2022] DELETE /identity/v3/projects/a23b541a4f0b45dd876920f17860226a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:59.916325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cd893-ec69-4720-9d29-44ba88678c1e None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:59.916756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cd893-ec69-4720-9d29-44ba88678c1e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:59.916902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cd893-ec69-4720-9d29-44ba88678c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:59.936490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-561cd893-ec69-4720-9d29-44ba88678c1e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:59.937751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-561cd893-ec69-4720-9d29-44ba88678c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:00.007701 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1001/1993] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:00.015406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f95d0-7687-4e89-98fb-7bb89d9e4b2a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.015956 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f95d0-7687-4e89-98fb-7bb89d9e4b2a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.021147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f95d0-7687-4e89-98fb-7bb89d9e4b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.045500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-8d3f95d0-7687-4e89-98fb-7bb89d9e4b2a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:00.046764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8d3f95d0-7687-4e89-98fb-7bb89d9e4b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:00.056618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 993/1994] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:00.061638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cd893-ec69-4720-9d29-44ba88678c1e None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:00.073512 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cd893-ec69-4720-9d29-44ba88678c1e None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:00.079167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2622b-0fa4-4285-8250-15c70afe4640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:00.079904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2622b-0fa4-4285-8250-15c70afe4640 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.080359 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2622b-0fa4-4285-8250-15c70afe4640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.080638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2622b-0fa4-4285-8250-15c70afe4640 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.081337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2622b-0fa4-4285-8250-15c70afe4640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:18:00.081645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2622b-0fa4-4285-8250-15c70afe4640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:00.083265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2622b-0fa4-4285-8250-15c70afe4640 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:00.099245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1002/1995] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:18:00.105073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f95d0-7687-4e89-98fb-7bb89d9e4b2a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:00.118566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f95d0-7687-4e89-98fb-7bb89d9e4b2a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:00.123562 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0db9e2-1141-41f5-ac36-d5b48b2bfcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:00.124180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0db9e2-1141-41f5-ac36-d5b48b2bfcc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.124462 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0db9e2-1141-41f5-ac36-d5b48b2bfcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.124738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0db9e2-1141-41f5-ac36-d5b48b2bfcc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.125437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0db9e2-1141-41f5-ac36-d5b48b2bfcc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:00.127448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0db9e2-1141-41f5-ac36-d5b48b2bfcc3 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:00.261278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 994/1996] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:00 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:00.267478 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2622b-0fa4-4285-8250-15c70afe4640 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:00.278711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2622b-0fa4-4285-8250-15c70afe4640 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:00.298941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0db9e2-1141-41f5-ac36-d5b48b2bfcc3 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:00.300700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3f0db9e2-1141-41f5-ac36-d5b48b2bfcc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:00.355900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8ec3-275d-4aea-aaa5-2b09654e2193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:00.355900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8ec3-275d-4aea-aaa5-2b09654e2193 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.355900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8ec3-275d-4aea-aaa5-2b09654e2193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.355900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8ec3-275d-4aea-aaa5-2b09654e2193 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.356325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e8ec3-275d-4aea-aaa5-2b09654e2193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:00.357172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e8ec3-275d-4aea-aaa5-2b09654e2193 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:00.377835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0db9e2-1141-41f5-ac36-d5b48b2bfcc3 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:00.384730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:00.384730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.384730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.384730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] PATH_INFO: `/v3/projects/4e3b431223ce4441b07437c7b7ecb34c` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.385605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e3b431223ce4441b07437c7b7ecb34c)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:00.387531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:00.458866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:00.465613 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1003/1997] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:00 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:00.470029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8ec3-275d-4aea-aaa5-2b09654e2193 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:00.479860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8ec3-275d-4aea-aaa5-2b09654e2193 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:00.484736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d575e-56a4-4d44-a13d-9f3d93d5b8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:00.485366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d575e-56a4-4d44-a13d-9f3d93d5b8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.485651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d575e-56a4-4d44-a13d-9f3d93d5b8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.485928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d575e-56a4-4d44-a13d-9f3d93d5b8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.493481 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d575e-56a4-4d44-a13d-9f3d93d5b8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:00.493481 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d575e-56a4-4d44-a13d-9f3d93d5b8f5 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:00.498766 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1004/1998] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:18:00.503262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d575e-56a4-4d44-a13d-9f3d93d5b8f5 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:00.514641 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d575e-56a4-4d44-a13d-9f3d93d5b8f5 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:00.519685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646bffa-444d-4574-8369-1847253cf9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:00.520283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646bffa-444d-4574-8369-1847253cf9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.520569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646bffa-444d-4574-8369-1847253cf9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.520845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646bffa-444d-4574-8369-1847253cf9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.521524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646bffa-444d-4574-8369-1847253cf9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:00.523144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646bffa-444d-4574-8369-1847253cf9d7 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:00.532031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e3b431223ce4441b07437c7b7ecb34c', 'request_id': 'req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:00.534033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1005/1999] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:18:00.538073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646bffa-444d-4574-8369-1847253cf9d7 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:00.540769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:00.549740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e3b431223ce4441b07437c7b7ecb34c', 'request_id': 'req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:00.549740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e3b431223ce4441b07437c7b7ecb34c', 'request_id': 'req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:00.549992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646bffa-444d-4574-8369-1847253cf9d7 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:00.607293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf56415-ecb3-4efe-a5ca-4751ede92b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:00.607891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf56415-ecb3-4efe-a5ca-4751ede92b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.608179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf56415-ecb3-4efe-a5ca-4751ede92b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.608456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf56415-ecb3-4efe-a5ca-4751ede92b5b admin admin] PATH_INFO: `/v3/projects/915739ae2ca54f4ca1d21bfbcbcb725a/users/983902e85e734e9096ef8b579f907496/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.624213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf56415-ecb3-4efe-a5ca-4751ede92b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=915739ae2ca54f4ca1d21bfbcbcb725a, user_id=983902e85e734e9096ef8b579f907496, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:00.626917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf56415-ecb3-4efe-a5ca-4751ede92b5b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:00.630216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] The token cache is being invalidate because project 4e3b431223ce4441b07437c7b7ecb34c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:00.630514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:00.631191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0156ee60-2c6f-4fa0-ae2f-8f538f4b442b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:00.632822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 995/2000] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:00 2022] DELETE /identity/v3/projects/4e3b431223ce4441b07437c7b7ecb34c => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:00.638446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06cc32-b808-4eda-ba46-48d16e78928f None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.638736 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06cc32-b808-4eda-ba46-48d16e78928f None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.639017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06cc32-b808-4eda-ba46-48d16e78928f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.658883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1b06cc32-b808-4eda-ba46-48d16e78928f None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:00.660093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1b06cc32-b808-4eda-ba46-48d16e78928f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:00.669037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 996/2001] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:00.676677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10321317-b33c-4157-8aa6-b43cc42a3a6c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.677008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10321317-b33c-4157-8aa6-b43cc42a3a6c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.677287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10321317-b33c-4157-8aa6-b43cc42a3a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.697553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-10321317-b33c-4157-8aa6-b43cc42a3a6c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:00.698746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-10321317-b33c-4157-8aa6-b43cc42a3a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:00.707200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 997/2002] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:00.712065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10321317-b33c-4157-8aa6-b43cc42a3a6c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:00.721664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10321317-b33c-4157-8aa6-b43cc42a3a6c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:00.721947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1006/2003] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:00 2022] PUT /identity/v3/projects/915739ae2ca54f4ca1d21bfbcbcb725a/users/983902e85e734e9096ef8b579f907496/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:00.733228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d366f7-8d38-4957-ae69-711c9c23982e None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.734050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d366f7-8d38-4957-ae69-711c9c23982e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.734050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d366f7-8d38-4957-ae69-711c9c23982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.763624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-55d366f7-8d38-4957-ae69-711c9c23982e None None] MFA Rules not processed for user `983902e85e734e9096ef8b579f907496`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:00.765055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-55d366f7-8d38-4957-ae69-711c9c23982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:00.794276 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0857f-580c-4ccc-a4fc-e0eed4b69d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:00.794905 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0857f-580c-4ccc-a4fc-e0eed4b69d42 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.795458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0857f-580c-4ccc-a4fc-e0eed4b69d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.795458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0857f-580c-4ccc-a4fc-e0eed4b69d42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.796176 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0857f-580c-4ccc-a4fc-e0eed4b69d42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:18:00.796710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0857f-580c-4ccc-a4fc-e0eed4b69d42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:00.798435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0857f-580c-4ccc-a4fc-e0eed4b69d42 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:00.812371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 998/2004] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:18:00.816722 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0857f-580c-4ccc-a4fc-e0eed4b69d42 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:00.827131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0857f-580c-4ccc-a4fc-e0eed4b69d42 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:00.832507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0600258-8548-41c5-9cb6-fd5fc7dfbd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:00.833156 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0600258-8548-41c5-9cb6-fd5fc7dfbd6a admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.833438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0600258-8548-41c5-9cb6-fd5fc7dfbd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.833714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0600258-8548-41c5-9cb6-fd5fc7dfbd6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.834347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0600258-8548-41c5-9cb6-fd5fc7dfbd6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:00.836920 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0600258-8548-41c5-9cb6-fd5fc7dfbd6a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:00.838840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1007/2005] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:00.850694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d366f7-8d38-4957-ae69-711c9c23982e None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:00.852978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-55d366f7-8d38-4957-ae69-711c9c23982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:00.918374 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d366f7-8d38-4957-ae69-711c9c23982e None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:00.922174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872a4a6-c238-4eab-9bfc-1e8f5915b6f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:00.922796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872a4a6-c238-4eab-9bfc-1e8f5915b6f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:00.923173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872a4a6-c238-4eab-9bfc-1e8f5915b6f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:00.923379 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872a4a6-c238-4eab-9bfc-1e8f5915b6f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:00.933863 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872a4a6-c238-4eab-9bfc-1e8f5915b6f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:00.935940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872a4a6-c238-4eab-9bfc-1e8f5915b6f8 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:00.946188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1008/2006] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:00.970456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 999/2007] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:00 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:00.975436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872a4a6-c238-4eab-9bfc-1e8f5915b6f8 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:00.992898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872a4a6-c238-4eab-9bfc-1e8f5915b6f8 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:01.055762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:01.072789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f25683-4cca-4694-9594-3ddf472365a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:01.073559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f25683-4cca-4694-9594-3ddf472365a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:01.073905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f25683-4cca-4694-9594-3ddf472365a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:01.074245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f25683-4cca-4694-9594-3ddf472365a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:01.075047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f25683-4cca-4694-9594-3ddf472365a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:01.077097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f25683-4cca-4694-9594-3ddf472365a5 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:01.207991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1009/2008] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:00 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:01.212507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0600258-8548-41c5-9cb6-fd5fc7dfbd6a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:01.222958 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0600258-8548-41c5-9cb6-fd5fc7dfbd6a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:01.228246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1fa43-2f05-486d-ac01-ebf701dfba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:01.228984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1fa43-2f05-486d-ac01-ebf701dfba80 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:01.229310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1fa43-2f05-486d-ac01-ebf701dfba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:01.229582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1fa43-2f05-486d-ac01-ebf701dfba80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:01.230221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1fa43-2f05-486d-ac01-ebf701dfba80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:01.231867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1fa43-2f05-486d-ac01-ebf701dfba80 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:01.243709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1000/2009] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:01.247717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f25683-4cca-4694-9594-3ddf472365a5 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:01.260450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f25683-4cca-4694-9594-3ddf472365a5 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:01.267785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19395a89-4118-453e-b283-12fd9b826d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:01.268892 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19395a89-4118-453e-b283-12fd9b826d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:01.269337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19395a89-4118-453e-b283-12fd9b826d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:01.269607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19395a89-4118-453e-b283-12fd9b826d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:01.270266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19395a89-4118-453e-b283-12fd9b826d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:01.278384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19395a89-4118-453e-b283-12fd9b826d4c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:01.284638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1010/2010] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:01.293637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1fa43-2f05-486d-ac01-ebf701dfba80 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:01.304111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1fa43-2f05-486d-ac01-ebf701dfba80 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:01.312393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cded3bd-b078-4ae8-b458-696854defff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:01.312393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cded3bd-b078-4ae8-b458-696854defff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:01.312393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cded3bd-b078-4ae8-b458-696854defff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:01.312393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cded3bd-b078-4ae8-b458-696854defff4 admin admin] PATH_INFO: `/v3/projects/60b41069ee7a4e678ef544404221f65d/users/1d44d26266834112834c1ee9ae97f4d0/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:01.332262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cded3bd-b078-4ae8-b458-696854defff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b41069ee7a4e678ef544404221f65d, user_id=1d44d26266834112834c1ee9ae97f4d0, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:01.335005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cded3bd-b078-4ae8-b458-696854defff4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:01.537600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1001/2011] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:01 2022] PUT /identity/v3/projects/60b41069ee7a4e678ef544404221f65d/users/1d44d26266834112834c1ee9ae97f4d0/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:01.550341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bfe6f-56a7-45a4-bbb8-75d02897098b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:01.550636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bfe6f-56a7-45a4-bbb8-75d02897098b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:01.550919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bfe6f-56a7-45a4-bbb8-75d02897098b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:01.582680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-935bfe6f-56a7-45a4-bbb8-75d02897098b None None] MFA Rules not processed for user `1d44d26266834112834c1ee9ae97f4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:01.583927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-935bfe6f-56a7-45a4-bbb8-75d02897098b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:01.655717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1011/2012] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:01.668025 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cded3bd-b078-4ae8-b458-696854defff4 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:01.670002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6cded3bd-b078-4ae8-b458-696854defff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:01.732221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cded3bd-b078-4ae8-b458-696854defff4 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:01.735760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69190c-0714-43c6-850a-c799ebdc97ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:01.736358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69190c-0714-43c6-850a-c799ebdc97ad service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:01.736636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69190c-0714-43c6-850a-c799ebdc97ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:01.736933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69190c-0714-43c6-850a-c799ebdc97ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:01.745526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69190c-0714-43c6-850a-c799ebdc97ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:01.746943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69190c-0714-43c6-850a-c799ebdc97ad service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:01.755619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1002/2013] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:02.120590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:03.910390 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:04.648083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bfe6f-56a7-45a4-bbb8-75d02897098b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:04.649566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-935bfe6f-56a7-45a4-bbb8-75d02897098b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:04.716185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bfe6f-56a7-45a4-bbb8-75d02897098b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:04.721541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:04.722163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:04.722438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:04.722710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] PATH_INFO: `/v3/users/983902e85e734e9096ef8b579f907496` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:04.723853 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=983902e85e734e9096ef8b579f907496)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:04.725799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:04.993249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '983902e85e734e9096ef8b579f907496'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:04.994465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '983902e85e734e9096ef8b579f907496'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:05.001933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '983902e85e734e9096ef8b579f907496'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:05.007393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:05.015277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '983902e85e734e9096ef8b579f907496'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:05.173160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '983902e85e734e9096ef8b579f907496'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:05.181269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1012/2014] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:04 2022] DELETE /identity/v3/users/983902e85e734e9096ef8b579f907496 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:05.187452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb17bd2-610e-4afb-8e59-4c4b3ec95503 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:05.187751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb17bd2-610e-4afb-8e59-4c4b3ec95503 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:05.188039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb17bd2-610e-4afb-8e59-4c4b3ec95503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:05.210221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6fb17bd2-610e-4afb-8e59-4c4b3ec95503 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:05.212084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6fb17bd2-610e-4afb-8e59-4c4b3ec95503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:05.270389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1003/2015] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:05.281631 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:05.283072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:05.344443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb28f-601d-4d4d-92c6-fac9718731ba admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:05.348182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8946339-520f-461a-9043-9db9fb80cbe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:05.348791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8946339-520f-461a-9043-9db9fb80cbe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:05.349100 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8946339-520f-461a-9043-9db9fb80cbe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:05.349377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8946339-520f-461a-9043-9db9fb80cbe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:05.359141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8946339-520f-461a-9043-9db9fb80cbe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:05.360507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8946339-520f-461a-9043-9db9fb80cbe8 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:05.370288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1013/2016] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:05.917594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb17bd2-610e-4afb-8e59-4c4b3ec95503 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:05.919198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6fb17bd2-610e-4afb-8e59-4c4b3ec95503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:05.933806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb17bd2-610e-4afb-8e59-4c4b3ec95503 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:05.938804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:05.939451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:05.939735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:05.940012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] PATH_INFO: `/v3/projects/915739ae2ca54f4ca1d21bfbcbcb725a` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:05.941193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=915739ae2ca54f4ca1d21bfbcbcb725a)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:05.943216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:06.072268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '915739ae2ca54f4ca1d21bfbcbcb725a', 'request_id': 'req-7882af74-a75f-4ece-9a1a-a816cc2be8d3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:06.077777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '915739ae2ca54f4ca1d21bfbcbcb725a', 'request_id': 'req-7882af74-a75f-4ece-9a1a-a816cc2be8d3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:06.084631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '915739ae2ca54f4ca1d21bfbcbcb725a', 'request_id': 'req-7882af74-a75f-4ece-9a1a-a816cc2be8d3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:06.195524 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] The token cache is being invalidate because project 915739ae2ca54f4ca1d21bfbcbcb725a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:06.195888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:06.196843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7882af74-a75f-4ece-9a1a-a816cc2be8d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:06.199528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1004/2017] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:05 2022] DELETE /identity/v3/projects/915739ae2ca54f4ca1d21bfbcbcb725a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:06.204693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b660c7-d395-46da-9b75-b1cc7b081493 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:06.205200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b660c7-d395-46da-9b75-b1cc7b081493 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:06.205312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b660c7-d395-46da-9b75-b1cc7b081493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:06.226822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-a7b660c7-d395-46da-9b75-b1cc7b081493 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:06.229097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a7b660c7-d395-46da-9b75-b1cc7b081493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:06.291033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1014/2018] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:06.298972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d752d-7d07-49fd-99bf-fc666df56616 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:06.299272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d752d-7d07-49fd-99bf-fc666df56616 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:06.299557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d752d-7d07-49fd-99bf-fc666df56616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:06.310483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:06.320082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-801d752d-7d07-49fd-99bf-fc666df56616 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:06.321444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-801d752d-7d07-49fd-99bf-fc666df56616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:06.329634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1005/2019] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:06.334085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b660c7-d395-46da-9b75-b1cc7b081493 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:06.344207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b660c7-d395-46da-9b75-b1cc7b081493 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:06.348826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2913e-5a54-41d0-a9f9-ad9b499f3b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:06.349464 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2913e-5a54-41d0-a9f9-ad9b499f3b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:06.349746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2913e-5a54-41d0-a9f9-ad9b499f3b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:06.350022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2913e-5a54-41d0-a9f9-ad9b499f3b9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:06.350671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2913e-5a54-41d0-a9f9-ad9b499f3b9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:18:06.350972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2913e-5a54-41d0-a9f9-ad9b499f3b9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:06.353132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2913e-5a54-41d0-a9f9-ad9b499f3b9a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:06.365013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1015/2020] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:18:06.370498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d752d-7d07-49fd-99bf-fc666df56616 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:06.379922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d752d-7d07-49fd-99bf-fc666df56616 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:06.385024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314627d6-64c3-44da-bde0-a5fd70b49603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:06.386174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314627d6-64c3-44da-bde0-a5fd70b49603 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:06.386466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314627d6-64c3-44da-bde0-a5fd70b49603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:06.386748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314627d6-64c3-44da-bde0-a5fd70b49603 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:06.387388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314627d6-64c3-44da-bde0-a5fd70b49603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:06.389371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314627d6-64c3-44da-bde0-a5fd70b49603 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:06.478209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1006/2021] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:06 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:06.482360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2913e-5a54-41d0-a9f9-ad9b499f3b9a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:06.493313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2913e-5a54-41d0-a9f9-ad9b499f3b9a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:06.556901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90e882-586a-4add-9eea-c0f9fc9e9023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:06.557523 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90e882-586a-4add-9eea-c0f9fc9e9023 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:06.557856 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90e882-586a-4add-9eea-c0f9fc9e9023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:06.558106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90e882-586a-4add-9eea-c0f9fc9e9023 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:06.558743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90e882-586a-4add-9eea-c0f9fc9e9023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:06.561044 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90e882-586a-4add-9eea-c0f9fc9e9023 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:06.751023 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1016/2022] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:06 2022] POST /identity/v3/users => generated 286 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:06.751138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314627d6-64c3-44da-bde0-a5fd70b49603 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:06.751138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314627d6-64c3-44da-bde0-a5fd70b49603 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:06.752424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d274004-351f-4de8-8e92-57170e85a577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:06.753062 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d274004-351f-4de8-8e92-57170e85a577 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:06.753346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d274004-351f-4de8-8e92-57170e85a577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:06.753660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d274004-351f-4de8-8e92-57170e85a577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:06.754299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d274004-351f-4de8-8e92-57170e85a577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:06.755951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d274004-351f-4de8-8e92-57170e85a577 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:06.767559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1007/2023] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:06.771875 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90e882-586a-4add-9eea-c0f9fc9e9023 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:06.782147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90e882-586a-4add-9eea-c0f9fc9e9023 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:06.786924 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1f6b7-48c3-40ef-a1f7-e989b4b1831f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:06.787525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1f6b7-48c3-40ef-a1f7-e989b4b1831f admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:06.787807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1f6b7-48c3-40ef-a1f7-e989b4b1831f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:06.788084 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1f6b7-48c3-40ef-a1f7-e989b4b1831f admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:06.788886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1f6b7-48c3-40ef-a1f7-e989b4b1831f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:06.790596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1f6b7-48c3-40ef-a1f7-e989b4b1831f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:06.801383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1017/2024] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:06.806629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d274004-351f-4de8-8e92-57170e85a577 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:06.816800 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d274004-351f-4de8-8e92-57170e85a577 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:06.880971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b923d2f-b5db-4e67-ad19-d65c3e00c8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:06.880971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b923d2f-b5db-4e67-ad19-d65c3e00c8ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:06.880971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b923d2f-b5db-4e67-ad19-d65c3e00c8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:06.880971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b923d2f-b5db-4e67-ad19-d65c3e00c8ae admin admin] PATH_INFO: `/v3/projects/c853841b06da473094805c56c4281174/users/beff0559f8c0471aa4a6496f02576442/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:06.883157 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b923d2f-b5db-4e67-ad19-d65c3e00c8ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c853841b06da473094805c56c4281174, user_id=beff0559f8c0471aa4a6496f02576442, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:06.883157 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b923d2f-b5db-4e67-ad19-d65c3e00c8ae admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:07.123405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1008/2025] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:06 2022] PUT /identity/v3/projects/c853841b06da473094805c56c4281174/users/beff0559f8c0471aa4a6496f02576442/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:07.134092 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10414774-9b35-4760-912e-e3cf9735f71a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:07.134383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10414774-9b35-4760-912e-e3cf9735f71a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:07.134664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10414774-9b35-4760-912e-e3cf9735f71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:07.162058 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-10414774-9b35-4760-912e-e3cf9735f71a None None] MFA Rules not processed for user `beff0559f8c0471aa4a6496f02576442`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:07.163283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-10414774-9b35-4760-912e-e3cf9735f71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:07.230161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1018/2026] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:07.241838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b923d2f-b5db-4e67-ad19-d65c3e00c8ae admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:07.244176 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7b923d2f-b5db-4e67-ad19-d65c3e00c8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:07.271741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10414774-9b35-4760-912e-e3cf9735f71a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:07.273199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-10414774-9b35-4760-912e-e3cf9735f71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:07.328373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:07.329513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b923d2f-b5db-4e67-ad19-d65c3e00c8ae admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:07.333397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2549ed0-f4db-419a-8d06-dabad6f18cde service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0b124ca267b4a4391557ad1396524cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:07.334009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2549ed0-f4db-419a-8d06-dabad6f18cde service glance] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:07.334293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2549ed0-f4db-419a-8d06-dabad6f18cde service glance] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:07.334572 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2549ed0-f4db-419a-8d06-dabad6f18cde service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:07.341405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10414774-9b35-4760-912e-e3cf9735f71a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:07.344138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2549ed0-f4db-419a-8d06-dabad6f18cde service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:07.345612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2549ed0-f4db-419a-8d06-dabad6f18cde service glance] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:07.345790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:07.346448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:07.346736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:07.347065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] PATH_INFO: `/v3/users/f1f654b76e8a4f1081800e0ddd3a552b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:07.348179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1f654b76e8a4f1081800e0ddd3a552b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:07.350148 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:07.356119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1009/2027] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:07.558520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1f654b76e8a4f1081800e0ddd3a552b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:07.565170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1f654b76e8a4f1081800e0ddd3a552b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:07.566652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1f654b76e8a4f1081800e0ddd3a552b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:07.578775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1f654b76e8a4f1081800e0ddd3a552b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:07.664744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1f654b76e8a4f1081800e0ddd3a552b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:07.671277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1019/2028] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:07 2022] DELETE /identity/v3/users/f1f654b76e8a4f1081800e0ddd3a552b => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:07.676193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c11e3-889c-459b-898d-5b1ac12bdfe5 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:07.676559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c11e3-889c-459b-898d-5b1ac12bdfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:07.676900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c11e3-889c-459b-898d-5b1ac12bdfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:07.696256 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-d96c11e3-889c-459b-898d-5b1ac12bdfe5 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:07.697601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d96c11e3-889c-459b-898d-5b1ac12bdfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:07.755813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1010/2029] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:07.766972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:07.768262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:07.825958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5235df1-9288-4a52-a5db-ee3fbee3c6aa admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:07.829617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a25fc8-184c-49ab-a037-53ef3f0f6993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:07.830225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25fc8-184c-49ab-a037-53ef3f0f6993 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:07.830509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25fc8-184c-49ab-a037-53ef3f0f6993 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:07.830784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25fc8-184c-49ab-a037-53ef3f0f6993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:07.840638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a25fc8-184c-49ab-a037-53ef3f0f6993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:07.843893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a25fc8-184c-49ab-a037-53ef3f0f6993 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:07.851766 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1020/2030] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:07.959980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:08.059355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:08.368066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c11e3-889c-459b-898d-5b1ac12bdfe5 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:08.375862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d96c11e3-889c-459b-898d-5b1ac12bdfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:08.383523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c11e3-889c-459b-898d-5b1ac12bdfe5 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:08.388793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:08.389489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:08.389749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:08.390226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] PATH_INFO: `/v3/projects/419d159f16d645519676003ef0af8e6e` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:08.391588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=419d159f16d645519676003ef0af8e6e)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:08.394310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:08.560077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '419d159f16d645519676003ef0af8e6e', 'request_id': 'req-a76cb49d-273d-43dd-a14c-045be373fee8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:08.565930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '419d159f16d645519676003ef0af8e6e', 'request_id': 'req-a76cb49d-273d-43dd-a14c-045be373fee8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:08.575027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '419d159f16d645519676003ef0af8e6e', 'request_id': 'req-a76cb49d-273d-43dd-a14c-045be373fee8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:08.661510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] The token cache is being invalidate because project 419d159f16d645519676003ef0af8e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:08.661510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:08.661510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:08.661510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1011/2031] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:08 2022] DELETE /identity/v3/projects/419d159f16d645519676003ef0af8e6e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:08.661810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a25fc8-184c-49ab-a037-53ef3f0f6993 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:08.672029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a25fc8-184c-49ab-a037-53ef3f0f6993 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:08.731881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9095c2-fd25-4a39-9e45-be8d1b86e6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:08.732851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9095c2-fd25-4a39-9e45-be8d1b86e6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:08.733230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9095c2-fd25-4a39-9e45-be8d1b86e6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:08.733507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9095c2-fd25-4a39-9e45-be8d1b86e6d7 admin admin] PATH_INFO: `/v3/users/a86cf31df3a94c63acb71852c73ce192` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:08.734635 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9095c2-fd25-4a39-9e45-be8d1b86e6d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a86cf31df3a94c63acb71852c73ce192)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:08.736603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9095c2-fd25-4a39-9e45-be8d1b86e6d7 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:08.970551 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3a9095c2-fd25-4a39-9e45-be8d1b86e6d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a86cf31df3a94c63acb71852c73ce192'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:08.971205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3a9095c2-fd25-4a39-9e45-be8d1b86e6d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a86cf31df3a94c63acb71852c73ce192'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:08.979871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3a9095c2-fd25-4a39-9e45-be8d1b86e6d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a86cf31df3a94c63acb71852c73ce192'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:08.995643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3a9095c2-fd25-4a39-9e45-be8d1b86e6d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a86cf31df3a94c63acb71852c73ce192'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:09.075522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3a9095c2-fd25-4a39-9e45-be8d1b86e6d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a86cf31df3a94c63acb71852c73ce192'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:09.081958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1021/2032] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:08 2022] DELETE /identity/v3/users/a86cf31df3a94c63acb71852c73ce192 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:09.200037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:09.299944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:09.639156 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:09.641715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:09.667112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cb49d-273d-43dd-a14c-045be373fee8 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:09.735251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:09.736287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:09.736733 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:09.737226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] PATH_INFO: `/v3/projects/b5fb56c3ecb940099d3121bdeb7fc984` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:09.740774 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5fb56c3ecb940099d3121bdeb7fc984)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:09.742987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:09.960459 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5fb56c3ecb940099d3121bdeb7fc984', 'request_id': 'req-399c580d-cecf-46da-81de-1177daeae42d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:09.972931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5fb56c3ecb940099d3121bdeb7fc984', 'request_id': 'req-399c580d-cecf-46da-81de-1177daeae42d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:09.982599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5fb56c3ecb940099d3121bdeb7fc984', 'request_id': 'req-399c580d-cecf-46da-81de-1177daeae42d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:10.067671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:10.110006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] The token cache is being invalidate because project b5fb56c3ecb940099d3121bdeb7fc984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:10.110394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:10.111108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-399c580d-cecf-46da-81de-1177daeae42d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:10.112653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1012/2033] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:09 2022] DELETE /identity/v3/projects/b5fb56c3ecb940099d3121bdeb7fc984 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:10.118612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab0897-658f-410a-9785-44d441f54c94 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:10.118902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab0897-658f-410a-9785-44d441f54c94 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:10.119180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab0897-658f-410a-9785-44d441f54c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:10.139978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3dab0897-658f-410a-9785-44d441f54c94 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:10.145418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3dab0897-658f-410a-9785-44d441f54c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:10.206896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1022/2034] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:10.217148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4cf0a-cdcb-41ec-9cf6-3bad652b7256 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:10.217466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4cf0a-cdcb-41ec-9cf6-3bad652b7256 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:10.217744 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4cf0a-cdcb-41ec-9cf6-3bad652b7256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:10.239692 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-5ea4cf0a-cdcb-41ec-9cf6-3bad652b7256 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:10.241284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5ea4cf0a-cdcb-41ec-9cf6-3bad652b7256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:10.253701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1013/2035] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:10.254951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab0897-658f-410a-9785-44d441f54c94 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:10.270010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab0897-658f-410a-9785-44d441f54c94 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:10.275601 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2bad-2585-4de1-8ee7-efee2c46f6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:10.276303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2bad-2585-4de1-8ee7-efee2c46f6ab admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:10.276653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2bad-2585-4de1-8ee7-efee2c46f6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:10.277030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2bad-2585-4de1-8ee7-efee2c46f6ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:10.277916 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2bad-2585-4de1-8ee7-efee2c46f6ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:18:10.278360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2bad-2585-4de1-8ee7-efee2c46f6ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:10.280384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2bad-2585-4de1-8ee7-efee2c46f6ab admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:10.295436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1023/2036] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:18:10.300454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea4cf0a-cdcb-41ec-9cf6-3bad652b7256 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:10.313556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea4cf0a-cdcb-41ec-9cf6-3bad652b7256 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:10.319965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d19a2-acb6-42ca-9e3b-7cfed377e453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:10.320651 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d19a2-acb6-42ca-9e3b-7cfed377e453 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:10.320974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d19a2-acb6-42ca-9e3b-7cfed377e453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:10.324240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d19a2-acb6-42ca-9e3b-7cfed377e453 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:10.324240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d19a2-acb6-42ca-9e3b-7cfed377e453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:10.324240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d19a2-acb6-42ca-9e3b-7cfed377e453 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:10.429269 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1014/2037] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:10 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:18:10.433850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2bad-2585-4de1-8ee7-efee2c46f6ab admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:10.443999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2bad-2585-4de1-8ee7-efee2c46f6ab admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:10.507395 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3711f21-8f40-4574-b216-bc82b5d456ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:10.508034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3711f21-8f40-4574-b216-bc82b5d456ef admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:10.508315 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3711f21-8f40-4574-b216-bc82b5d456ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:10.508804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3711f21-8f40-4574-b216-bc82b5d456ef admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:10.509581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3711f21-8f40-4574-b216-bc82b5d456ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:10.511695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3711f21-8f40-4574-b216-bc82b5d456ef admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:10.647737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1024/2038] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:10 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:10.652324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d19a2-acb6-42ca-9e3b-7cfed377e453 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:10.663192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d19a2-acb6-42ca-9e3b-7cfed377e453 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:10.669036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93b823-387f-4dae-bfaa-784ffbe18045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:10.669657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b823-387f-4dae-bfaa-784ffbe18045 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:10.669934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b823-387f-4dae-bfaa-784ffbe18045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:10.670205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b823-387f-4dae-bfaa-784ffbe18045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:10.670849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93b823-387f-4dae-bfaa-784ffbe18045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:10.672481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93b823-387f-4dae-bfaa-784ffbe18045 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:10.684147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1015/2039] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:18:10.688295 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3711f21-8f40-4574-b216-bc82b5d456ef admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:10.703096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3711f21-8f40-4574-b216-bc82b5d456ef admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:10.707816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874e6b5-b79b-4c12-8bae-b10e3d627c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:10.708428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874e6b5-b79b-4c12-8bae-b10e3d627c72 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:10.708789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874e6b5-b79b-4c12-8bae-b10e3d627c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:10.709210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874e6b5-b79b-4c12-8bae-b10e3d627c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:10.710052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874e6b5-b79b-4c12-8bae-b10e3d627c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:10.712801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874e6b5-b79b-4c12-8bae-b10e3d627c72 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:10.724332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1025/2040] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:18:10.729474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93b823-387f-4dae-bfaa-784ffbe18045 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:10.739156 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93b823-387f-4dae-bfaa-784ffbe18045 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:10.744798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3e8ce-23a7-45b7-bec5-b754386cc99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:10.746816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3e8ce-23a7-45b7-bec5-b754386cc99e admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:10.746816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3e8ce-23a7-45b7-bec5-b754386cc99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:10.746816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3e8ce-23a7-45b7-bec5-b754386cc99e admin admin] PATH_INFO: `/v3/projects/aa1ff79ba46a404fbeb9ad5608ebf9c5/users/f358f7731d334312b32f2b34b7b5209b/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:10.765151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3e8ce-23a7-45b7-bec5-b754386cc99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa1ff79ba46a404fbeb9ad5608ebf9c5, user_id=f358f7731d334312b32f2b34b7b5209b, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:10.767879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3e8ce-23a7-45b7-bec5-b754386cc99e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:10.817270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:10.881519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1016/2041] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:10 2022] PUT /identity/v3/projects/aa1ff79ba46a404fbeb9ad5608ebf9c5/users/f358f7731d334312b32f2b34b7b5209b/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:10.892759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bfaa5-8603-4c01-8260-a233f22a76c9 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:10.893079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bfaa5-8603-4c01-8260-a233f22a76c9 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:10.893372 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bfaa5-8603-4c01-8260-a233f22a76c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:10.923051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-141bfaa5-8603-4c01-8260-a233f22a76c9 None None] MFA Rules not processed for user `f358f7731d334312b32f2b34b7b5209b`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:10.924336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-141bfaa5-8603-4c01-8260-a233f22a76c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:11.000707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1026/2042] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:11.005174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3e8ce-23a7-45b7-bec5-b754386cc99e admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:11.090718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3e8ce-23a7-45b7-bec5-b754386cc99e admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:11.095759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579e244-69f2-4b23-85c3-6e13d7e921c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:11.096365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579e244-69f2-4b23-85c3-6e13d7e921c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:11.096645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579e244-69f2-4b23-85c3-6e13d7e921c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:11.096951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579e244-69f2-4b23-85c3-6e13d7e921c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:11.097577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579e244-69f2-4b23-85c3-6e13d7e921c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:11.099487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579e244-69f2-4b23-85c3-6e13d7e921c1 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:11.191973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1017/2043] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:11 2022] POST /identity/v3/projects => generated 389 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:11.197306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bfaa5-8603-4c01-8260-a233f22a76c9 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:11.209941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bfaa5-8603-4c01-8260-a233f22a76c9 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:11.270759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a581b87-0042-4dd6-9d39-fe8d21a380c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:11.271293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a581b87-0042-4dd6-9d39-fe8d21a380c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:11.271576 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a581b87-0042-4dd6-9d39-fe8d21a380c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:11.271879 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a581b87-0042-4dd6-9d39-fe8d21a380c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:11.272518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a581b87-0042-4dd6-9d39-fe8d21a380c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:11.274967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a581b87-0042-4dd6-9d39-fe8d21a380c8 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:11.419631 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1027/2044] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:11 2022] POST /identity/v3/users => generated 301 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:18:11.424776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:11.426173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579e244-69f2-4b23-85c3-6e13d7e921c1 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:11.437484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579e244-69f2-4b23-85c3-6e13d7e921c1 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:11.442239 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd84b75-01e1-4b2a-a9fc-e317d3147700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:11.442889 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd84b75-01e1-4b2a-a9fc-e317d3147700 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:11.443177 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd84b75-01e1-4b2a-a9fc-e317d3147700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:11.443502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd84b75-01e1-4b2a-a9fc-e317d3147700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:11.444128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd84b75-01e1-4b2a-a9fc-e317d3147700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:11.445847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd84b75-01e1-4b2a-a9fc-e317d3147700 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:11.457282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1018/2045] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:18:11.462111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a581b87-0042-4dd6-9d39-fe8d21a380c8 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:11.472881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a581b87-0042-4dd6-9d39-fe8d21a380c8 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:11.478629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b6b86-f262-424f-8626-d011ba7b2f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:11.483439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b6b86-f262-424f-8626-d011ba7b2f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:11.483439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b6b86-f262-424f-8626-d011ba7b2f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:11.483439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b6b86-f262-424f-8626-d011ba7b2f99 admin admin] PATH_INFO: `/v3/projects/f94dba8e836c49318089e6c22f2f4b16/users/3b1d258fc578473f969c85d7a745b542/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:11.499483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b6b86-f262-424f-8626-d011ba7b2f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94dba8e836c49318089e6c22f2f4b16, user_id=3b1d258fc578473f969c85d7a745b542, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:11.502353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b6b86-f262-424f-8626-d011ba7b2f99 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:11.606945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1028/2046] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:11 2022] PUT /identity/v3/projects/f94dba8e836c49318089e6c22f2f4b16/users/3b1d258fc578473f969c85d7a745b542/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:11.610957 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd84b75-01e1-4b2a-a9fc-e317d3147700 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:11.622155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd84b75-01e1-4b2a-a9fc-e317d3147700 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:11.680718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67152a-ddd9-4b8f-a46d-72d63896983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:11.681418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67152a-ddd9-4b8f-a46d-72d63896983e admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:11.681666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67152a-ddd9-4b8f-a46d-72d63896983e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:11.681945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67152a-ddd9-4b8f-a46d-72d63896983e admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:11.682593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67152a-ddd9-4b8f-a46d-72d63896983e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:11.684240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67152a-ddd9-4b8f-a46d-72d63896983e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:11.695331 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1019/2047] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:11.699899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b6b86-f262-424f-8626-d011ba7b2f99 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:11.710864 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b6b86-f262-424f-8626-d011ba7b2f99 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:11.722229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be97ba-e901-46bd-8563-90652bfef9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:11.722229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be97ba-e901-46bd-8563-90652bfef9e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:11.722229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be97ba-e901-46bd-8563-90652bfef9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:11.722229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be97ba-e901-46bd-8563-90652bfef9e8 admin admin] PATH_INFO: `/v3/projects/f94dba8e836c49318089e6c22f2f4b16/users/3b1d258fc578473f969c85d7a745b542/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:11.723971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be97ba-e901-46bd-8563-90652bfef9e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94dba8e836c49318089e6c22f2f4b16, user_id=3b1d258fc578473f969c85d7a745b542, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:11.728221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be97ba-e901-46bd-8563-90652bfef9e8 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:11.743528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1029/2048] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:11 2022] PUT /identity/v3/projects/f94dba8e836c49318089e6c22f2f4b16/users/3b1d258fc578473f969c85d7a745b542/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:11.754314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6d1e3-22a5-4c32-b05a-5d5a965fc91a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:11.761978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6d1e3-22a5-4c32-b05a-5d5a965fc91a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:11.761978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6d1e3-22a5-4c32-b05a-5d5a965fc91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:11.783998 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-aec6d1e3-22a5-4c32-b05a-5d5a965fc91a None None] MFA Rules not processed for user `3b1d258fc578473f969c85d7a745b542`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:11.785221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-aec6d1e3-22a5-4c32-b05a-5d5a965fc91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:11.857826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1020/2049] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:11 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:11.873245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be97ba-e901-46bd-8563-90652bfef9e8 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:11.873245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-71be97ba-e901-46bd-8563-90652bfef9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:11.936434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be97ba-e901-46bd-8563-90652bfef9e8 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:11.943902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4ef1b-b47f-48ac-a697-b0c97d874bc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:11.943902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4ef1b-b47f-48ac-a697-b0c97d874bc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:11.943902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4ef1b-b47f-48ac-a697-b0c97d874bc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:11.943902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4ef1b-b47f-48ac-a697-b0c97d874bc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:11.965820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4ef1b-b47f-48ac-a697-b0c97d874bc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:11.967294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4ef1b-b47f-48ac-a697-b0c97d874bc0 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:11.994144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1030/2050] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:11 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:12.171245 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6d1e3-22a5-4c32-b05a-5d5a965fc91a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:12.249293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6d1e3-22a5-4c32-b05a-5d5a965fc91a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:12.254290 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:12.254931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:12.255216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:12.255488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] PATH_INFO: `/v3/users/f358f7731d334312b32f2b34b7b5209b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:12.256715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f358f7731d334312b32f2b34b7b5209b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:12.259648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:12.430939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:12.508946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f358f7731d334312b32f2b34b7b5209b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:12.509685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f358f7731d334312b32f2b34b7b5209b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:12.519255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f358f7731d334312b32f2b34b7b5209b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:12.521669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f358f7731d334312b32f2b34b7b5209b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:12.599314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f358f7731d334312b32f2b34b7b5209b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:12.624592 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1021/2051] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:12 2022] DELETE /identity/v3/users/f358f7731d334312b32f2b34b7b5209b => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:12.631323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc23a1-fd2d-4b19-a5fe-7058ddba4a6d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:12.631979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc23a1-fd2d-4b19-a5fe-7058ddba4a6d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:12.633595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc23a1-fd2d-4b19-a5fe-7058ddba4a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:12.664182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-5fbc23a1-fd2d-4b19-a5fe-7058ddba4a6d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:12.665448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5fbc23a1-fd2d-4b19-a5fe-7058ddba4a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:12.726769 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1031/2052] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:12.736596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:12.738134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:12.801868 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6976b97-75b4-44f9-bfc5-7d42f515371b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:12.806142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890f71e-1751-4f70-8275-d70ae180072a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:12.806744 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890f71e-1751-4f70-8275-d70ae180072a service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:12.807016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890f71e-1751-4f70-8275-d70ae180072a service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:12.807284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890f71e-1751-4f70-8275-d70ae180072a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:12.816187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890f71e-1751-4f70-8275-d70ae180072a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:12.817619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890f71e-1751-4f70-8275-d70ae180072a service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:12.828508 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1022/2053] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:13.151497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:13.504250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc23a1-fd2d-4b19-a5fe-7058ddba4a6d None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:13.506421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5fbc23a1-fd2d-4b19-a5fe-7058ddba4a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:13.534151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc23a1-fd2d-4b19-a5fe-7058ddba4a6d None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:13.542228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:13.544778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:13.544778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:13.544778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] PATH_INFO: `/v3/projects/aa1ff79ba46a404fbeb9ad5608ebf9c5` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:13.546501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa1ff79ba46a404fbeb9ad5608ebf9c5)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:13.550181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:13.685774 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa1ff79ba46a404fbeb9ad5608ebf9c5', 'request_id': 'req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:13.691360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa1ff79ba46a404fbeb9ad5608ebf9c5', 'request_id': 'req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:13.761571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa1ff79ba46a404fbeb9ad5608ebf9c5', 'request_id': 'req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:13.774695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] The token cache is being invalidate because project aa1ff79ba46a404fbeb9ad5608ebf9c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:13.774985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:13.775650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:13.777746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1032/2054] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:13 2022] DELETE /identity/v3/projects/aa1ff79ba46a404fbeb9ad5608ebf9c5 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:13.788924 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890f71e-1751-4f70-8275-d70ae180072a service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:13.801025 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890f71e-1751-4f70-8275-d70ae180072a service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:13.861724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93a69e-8aa8-465b-98d4-4efb2aafeabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:13.862358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a69e-8aa8-465b-98d4-4efb2aafeabc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:13.862634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a69e-8aa8-465b-98d4-4efb2aafeabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:13.862907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a69e-8aa8-465b-98d4-4efb2aafeabc admin admin] PATH_INFO: `/v3/users/3b1d258fc578473f969c85d7a745b542` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:13.864374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93a69e-8aa8-465b-98d4-4efb2aafeabc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b1d258fc578473f969c85d7a745b542)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:13.866774 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93a69e-8aa8-465b-98d4-4efb2aafeabc admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:14.142234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ab93a69e-8aa8-465b-98d4-4efb2aafeabc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b1d258fc578473f969c85d7a745b542'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:14.142911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ab93a69e-8aa8-465b-98d4-4efb2aafeabc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b1d258fc578473f969c85d7a745b542'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:14.150625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ab93a69e-8aa8-465b-98d4-4efb2aafeabc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b1d258fc578473f969c85d7a745b542'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:14.151073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ab93a69e-8aa8-465b-98d4-4efb2aafeabc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b1d258fc578473f969c85d7a745b542'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:14.269674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ab93a69e-8aa8-465b-98d4-4efb2aafeabc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b1d258fc578473f969c85d7a745b542'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:14.288517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1023/2055] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:13 2022] DELETE /identity/v3/users/3b1d258fc578473f969c85d7a745b542 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:14.299959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:14.747115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:14.749894 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:14.784172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc88260-3e02-4b4b-818d-89d5ec7e8a00 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:14.838109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:14.850459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:14.851850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:14.852806 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:14.853016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] PATH_INFO: `/v3/projects/f94dba8e836c49318089e6c22f2f4b16` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:14.854477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f94dba8e836c49318089e6c22f2f4b16)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:14.856762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:15.033542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f94dba8e836c49318089e6c22f2f4b16', 'request_id': 'req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:15.040307 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f94dba8e836c49318089e6c22f2f4b16', 'request_id': 'req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:15.114343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:15.154344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f94dba8e836c49318089e6c22f2f4b16', 'request_id': 'req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:15.171381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] The token cache is being invalidate because project f94dba8e836c49318089e6c22f2f4b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:15.172401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:15.172401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b9c41fd0-69b7-4e2c-8a98-0cea94e980f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:15.173837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1033/2056] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:14 2022] DELETE /identity/v3/projects/f94dba8e836c49318089e6c22f2f4b16 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:15.181630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736adda3-a298-4405-a387-a7572c455374 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:15.182153 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736adda3-a298-4405-a387-a7572c455374 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:15.182651 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736adda3-a298-4405-a387-a7572c455374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:15.206322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-736adda3-a298-4405-a387-a7572c455374 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:15.207579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-736adda3-a298-4405-a387-a7572c455374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:15.276227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1024/2057] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:15.284158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7686331-a229-406d-a0a0-7f0a470a2bce None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:15.284436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7686331-a229-406d-a0a0-7f0a470a2bce None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:15.284710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7686331-a229-406d-a0a0-7f0a470a2bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:15.306682 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-c7686331-a229-406d-a0a0-7f0a470a2bce None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:15.307928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c7686331-a229-406d-a0a0-7f0a470a2bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:15.317061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1034/2058] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:15.323358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736adda3-a298-4405-a387-a7572c455374 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:15.333160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736adda3-a298-4405-a387-a7572c455374 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:15.341972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d2842-3ba3-4451-98a6-2df14afda519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:15.342600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d2842-3ba3-4451-98a6-2df14afda519 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:15.342883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d2842-3ba3-4451-98a6-2df14afda519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:15.343167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d2842-3ba3-4451-98a6-2df14afda519 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:15.343824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d2842-3ba3-4451-98a6-2df14afda519 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:18:15.344379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d2842-3ba3-4451-98a6-2df14afda519 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:15.346211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d2842-3ba3-4451-98a6-2df14afda519 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:15.358418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1025/2059] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:18:15.363707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7686331-a229-406d-a0a0-7f0a470a2bce None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:15.373553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7686331-a229-406d-a0a0-7f0a470a2bce None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:15.378239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d80e9f-0417-4d76-bb1c-550b6790f4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:15.379837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d80e9f-0417-4d76-bb1c-550b6790f4c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:15.379837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d80e9f-0417-4d76-bb1c-550b6790f4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:15.379837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d80e9f-0417-4d76-bb1c-550b6790f4c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:15.380089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d80e9f-0417-4d76-bb1c-550b6790f4c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:15.382540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d80e9f-0417-4d76-bb1c-550b6790f4c1 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:15.430507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:15.468321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1035/2060] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:15 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:18:15.469859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d2842-3ba3-4451-98a6-2df14afda519 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:15.483644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d2842-3ba3-4451-98a6-2df14afda519 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:15.568938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d67630-6298-40bd-b186-5d952b7aea05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:15.569773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d67630-6298-40bd-b186-5d952b7aea05 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:15.570166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d67630-6298-40bd-b186-5d952b7aea05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:15.571090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d67630-6298-40bd-b186-5d952b7aea05 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:15.572028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d67630-6298-40bd-b186-5d952b7aea05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:15.572902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:15.574241 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d67630-6298-40bd-b186-5d952b7aea05 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:15.670416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1026/2061] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:15 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:15.674902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d80e9f-0417-4d76-bb1c-550b6790f4c1 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:15.685961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d80e9f-0417-4d76-bb1c-550b6790f4c1 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:15.692548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f212d-d1f7-4ac0-99e1-60ac3fe63d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:15.693116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f212d-d1f7-4ac0-99e1-60ac3fe63d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:15.693617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f212d-d1f7-4ac0-99e1-60ac3fe63d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:15.694092 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f212d-d1f7-4ac0-99e1-60ac3fe63d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:15.700958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f212d-d1f7-4ac0-99e1-60ac3fe63d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:15.700958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f212d-d1f7-4ac0-99e1-60ac3fe63d7d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:15.710926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1036/2062] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:15.715535 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d67630-6298-40bd-b186-5d952b7aea05 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:15.727006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d67630-6298-40bd-b186-5d952b7aea05 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:15.731299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f5c33-64ec-442f-ba3a-e74f42c0fa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:15.731937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f5c33-64ec-442f-ba3a-e74f42c0fa38 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:15.732347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f5c33-64ec-442f-ba3a-e74f42c0fa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:15.732745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f5c33-64ec-442f-ba3a-e74f42c0fa38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:15.733376 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f5c33-64ec-442f-ba3a-e74f42c0fa38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:15.735146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f5c33-64ec-442f-ba3a-e74f42c0fa38 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:15.748200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1027/2063] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:15.755056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f212d-d1f7-4ac0-99e1-60ac3fe63d7d admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:15.768293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f212d-d1f7-4ac0-99e1-60ac3fe63d7d admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:15.775074 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37725-8d17-4b11-961d-83fe788809d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:15.775898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37725-8d17-4b11-961d-83fe788809d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:15.776235 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37725-8d17-4b11-961d-83fe788809d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:15.776592 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37725-8d17-4b11-961d-83fe788809d1 admin admin] PATH_INFO: `/v3/projects/4040505dee3440ecb273c5d4fb07a0d2/users/45c693ce56dd43f1be05223c81d8c49c/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:15.797289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b37725-8d17-4b11-961d-83fe788809d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4040505dee3440ecb273c5d4fb07a0d2, user_id=45c693ce56dd43f1be05223c81d8c49c, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:15.800126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b37725-8d17-4b11-961d-83fe788809d1 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:15.822740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:15.914097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1037/2064] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:15 2022] PUT /identity/v3/projects/4040505dee3440ecb273c5d4fb07a0d2/users/45c693ce56dd43f1be05223c81d8c49c/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:15.924719 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7ca4a-7097-414c-aeff-e10e22d18e7d None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:15.925385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7ca4a-7097-414c-aeff-e10e22d18e7d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:15.925385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7ca4a-7097-414c-aeff-e10e22d18e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:15.955884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a3c7ca4a-7097-414c-aeff-e10e22d18e7d None None] MFA Rules not processed for user `45c693ce56dd43f1be05223c81d8c49c`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:15.957172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a3c7ca4a-7097-414c-aeff-e10e22d18e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:16.038793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1028/2065] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:16.047186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37725-8d17-4b11-961d-83fe788809d1 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:16.049896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-49b37725-8d17-4b11-961d-83fe788809d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:16.133420 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37725-8d17-4b11-961d-83fe788809d1 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:16.137267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e30faa-7b23-44d3-beaa-25204ebe2a1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0b124ca267b4a4391557ad1396524cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:16.137883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e30faa-7b23-44d3-beaa-25204ebe2a1c service glance] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:16.138164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e30faa-7b23-44d3-beaa-25204ebe2a1c service glance] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:16.138436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e30faa-7b23-44d3-beaa-25204ebe2a1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:16.148327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e30faa-7b23-44d3-beaa-25204ebe2a1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:16.150565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e30faa-7b23-44d3-beaa-25204ebe2a1c service glance] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:16.162214 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:16.162513 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1038/2066] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:16.259619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:16.441198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:16.790495 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:16.932677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:17.544839 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:17.716065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:18.067130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:18.996816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:19.138171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:20.206158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:20.428236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:21.000386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:21.079046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:21.144580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:21.304685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:21.559154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:21.727274 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:21.749177 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75de8c-aa72-44d8-9065-8689e8353987 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:21.757441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75de8c-aa72-44d8-9065-8689e8353987 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:21.757441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75de8c-aa72-44d8-9065-8689e8353987 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:21.757441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1029/2067] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:18:21.763605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30caa49b-d0c3-4924-8fab-3a3247f21cdf None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:21.763966 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30caa49b-d0c3-4924-8fab-3a3247f21cdf None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:21.764250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30caa49b-d0c3-4924-8fab-3a3247f21cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:21.784561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-30caa49b-d0c3-4924-8fab-3a3247f21cdf None None] MFA Rules not processed for user `b1aa47a883604336a8ab90d324470477`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:21.785852 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-30caa49b-d0c3-4924-8fab-3a3247f21cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:21.838420 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:21.863045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1039/2068] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:18:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:21.874908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75de8c-aa72-44d8-9065-8689e8353987 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:21.876696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2f75de8c-aa72-44d8-9065-8689e8353987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:21.893897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75de8c-aa72-44d8-9065-8689e8353987 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:21.900023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebdb9e-5ee0-463d-95e2-137bef8c8f1d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:21.900646 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebdb9e-5ee0-463d-95e2-137bef8c8f1d service placement] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:21.901015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebdb9e-5ee0-463d-95e2-137bef8c8f1d service placement] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:21.901310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebdb9e-5ee0-463d-95e2-137bef8c8f1d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:21.910713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebdb9e-5ee0-463d-95e2-137bef8c8f1d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:21.912396 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebdb9e-5ee0-463d-95e2-137bef8c8f1d service placement] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:21.929656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1030/2069] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:21.934857 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30caa49b-d0c3-4924-8fab-3a3247f21cdf None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:21.937620 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-30caa49b-d0c3-4924-8fab-3a3247f21cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:22.039870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30caa49b-d0c3-4924-8fab-3a3247f21cdf None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:22.045527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48263dcd-c227-4af1-a428-7f8ecd09ffd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:22.046198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48263dcd-c227-4af1-a428-7f8ecd09ffd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:22.046500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48263dcd-c227-4af1-a428-7f8ecd09ffd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:22.046922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48263dcd-c227-4af1-a428-7f8ecd09ffd4 admin admin] PATH_INFO: `/v3/users/669c94a1591f41ae81d27f608987850f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:22.048050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48263dcd-c227-4af1-a428-7f8ecd09ffd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=669c94a1591f41ae81d27f608987850f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:22.050303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48263dcd-c227-4af1-a428-7f8ecd09ffd4 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:22.332250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-48263dcd-c227-4af1-a428-7f8ecd09ffd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '669c94a1591f41ae81d27f608987850f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:22.332967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-48263dcd-c227-4af1-a428-7f8ecd09ffd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '669c94a1591f41ae81d27f608987850f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:22.341954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-48263dcd-c227-4af1-a428-7f8ecd09ffd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '669c94a1591f41ae81d27f608987850f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:22.355122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-48263dcd-c227-4af1-a428-7f8ecd09ffd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '669c94a1591f41ae81d27f608987850f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:22.509645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-48263dcd-c227-4af1-a428-7f8ecd09ffd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '669c94a1591f41ae81d27f608987850f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:22.516412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1040/2070] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:21 2022] DELETE /identity/v3/users/669c94a1591f41ae81d27f608987850f => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:22.585854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:22.803378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebdb9e-5ee0-463d-95e2-137bef8c8f1d service placement] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:22.807050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:22.821586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebdb9e-5ee0-463d-95e2-137bef8c8f1d service placement] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:22.893962 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:22.894768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:22.895119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:22.895463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] PATH_INFO: `/v3/projects/d4b0c27ad44c455eb42d6a21931d42f8` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:22.897877 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4b0c27ad44c455eb42d6a21931d42f8)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:22.900374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:22.932591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:23.009515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4b0c27ad44c455eb42d6a21931d42f8', 'request_id': 'req-90b93f7f-9833-4d0d-826c-d2d42064612b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:23.016453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4b0c27ad44c455eb42d6a21931d42f8', 'request_id': 'req-90b93f7f-9833-4d0d-826c-d2d42064612b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:23.031967 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4b0c27ad44c455eb42d6a21931d42f8', 'request_id': 'req-90b93f7f-9833-4d0d-826c-d2d42064612b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:23.047051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:23.113511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] The token cache is being invalidate because project d4b0c27ad44c455eb42d6a21931d42f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:23.113839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:23.114464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-90b93f7f-9833-4d0d-826c-d2d42064612b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:23.117151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1031/2071] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:22 2022] DELETE /identity/v3/projects/d4b0c27ad44c455eb42d6a21931d42f8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:23.122395 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cb880-3bbb-41a4-a6d5-b631192cdb72 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:23.122682 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cb880-3bbb-41a4-a6d5-b631192cdb72 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:23.122978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cb880-3bbb-41a4-a6d5-b631192cdb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:23.144450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-2d3cb880-3bbb-41a4-a6d5-b631192cdb72 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:23.145707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2d3cb880-3bbb-41a4-a6d5-b631192cdb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:23.226898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1041/2072] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:23.234461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e509e1-6da9-40a1-a6da-e118e04480f9 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:23.234751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e509e1-6da9-40a1-a6da-e118e04480f9 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:23.235030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e509e1-6da9-40a1-a6da-e118e04480f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:23.259189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-68e509e1-6da9-40a1-a6da-e118e04480f9 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:23.260655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-68e509e1-6da9-40a1-a6da-e118e04480f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:23.269384 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1032/2073] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:23.274333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cb880-3bbb-41a4-a6d5-b631192cdb72 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:23.285800 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cb880-3bbb-41a4-a6d5-b631192cdb72 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:23.292658 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccd010-eaa2-457b-8ed9-1a3d37ba2fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:23.293326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccd010-eaa2-457b-8ed9-1a3d37ba2fbe admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:23.294108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccd010-eaa2-457b-8ed9-1a3d37ba2fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:23.294108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccd010-eaa2-457b-8ed9-1a3d37ba2fbe admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:23.294560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccd010-eaa2-457b-8ed9-1a3d37ba2fbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:18:23.294864 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccd010-eaa2-457b-8ed9-1a3d37ba2fbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:23.297892 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccd010-eaa2-457b-8ed9-1a3d37ba2fbe admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:23.313488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1042/2074] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:18:23.317919 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e509e1-6da9-40a1-a6da-e118e04480f9 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:23.328614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e509e1-6da9-40a1-a6da-e118e04480f9 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:23.334246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d351d4-bb7f-40b8-a97a-c459b13d1350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:23.334916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d351d4-bb7f-40b8-a97a-c459b13d1350 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:23.335204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d351d4-bb7f-40b8-a97a-c459b13d1350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:23.335464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d351d4-bb7f-40b8-a97a-c459b13d1350 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:23.336134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d351d4-bb7f-40b8-a97a-c459b13d1350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:23.338125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d351d4-bb7f-40b8-a97a-c459b13d1350 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:23.442435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1033/2075] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:23 2022] POST /identity/v3/projects => generated 389 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:23.446959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:23.448163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccd010-eaa2-457b-8ed9-1a3d37ba2fbe admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:23.458772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccd010-eaa2-457b-8ed9-1a3d37ba2fbe admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:23.518173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275f8fc-09b1-43d0-ac9c-17a594a25aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:23.518784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275f8fc-09b1-43d0-ac9c-17a594a25aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:23.519117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275f8fc-09b1-43d0-ac9c-17a594a25aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:23.519422 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275f8fc-09b1-43d0-ac9c-17a594a25aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:23.520057 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275f8fc-09b1-43d0-ac9c-17a594a25aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:23.522014 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275f8fc-09b1-43d0-ac9c-17a594a25aa0 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:23.527078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:23.613932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1043/2076] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:23 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:18:23.619853 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d351d4-bb7f-40b8-a97a-c459b13d1350 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:23.635631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d351d4-bb7f-40b8-a97a-c459b13d1350 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:23.640706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc955b-eed3-4533-b6eb-6445ed10b810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:23.642009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc955b-eed3-4533-b6eb-6445ed10b810 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:23.642009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc955b-eed3-4533-b6eb-6445ed10b810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:23.642009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc955b-eed3-4533-b6eb-6445ed10b810 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:23.642510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc955b-eed3-4533-b6eb-6445ed10b810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:23.644205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc955b-eed3-4533-b6eb-6445ed10b810 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:23.656924 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1034/2077] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:23.661514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275f8fc-09b1-43d0-ac9c-17a594a25aa0 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:23.672385 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275f8fc-09b1-43d0-ac9c-17a594a25aa0 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:23.677271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2e1a0-adf5-4cfe-929c-953e283e8167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:23.677881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2e1a0-adf5-4cfe-929c-953e283e8167 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:23.678154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2e1a0-adf5-4cfe-929c-953e283e8167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:23.678420 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2e1a0-adf5-4cfe-929c-953e283e8167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:23.679063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2e1a0-adf5-4cfe-929c-953e283e8167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:23.680679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2e1a0-adf5-4cfe-929c-953e283e8167 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:23.692357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1044/2078] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:23.698239 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc955b-eed3-4533-b6eb-6445ed10b810 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:23.714082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc955b-eed3-4533-b6eb-6445ed10b810 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:23.721072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582791b-52ff-4ed2-9095-e0ffb4cea07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:23.721780 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582791b-52ff-4ed2-9095-e0ffb4cea07d admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:23.722066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582791b-52ff-4ed2-9095-e0ffb4cea07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:23.722341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582791b-52ff-4ed2-9095-e0ffb4cea07d admin admin] PATH_INFO: `/v3/projects/dad90b6c7dd947dda6faa1fc1feb6d7b/users/8d72e2cc5b804b0bb556599a26d923ac/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:23.741093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582791b-52ff-4ed2-9095-e0ffb4cea07d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad90b6c7dd947dda6faa1fc1feb6d7b, user_id=8d72e2cc5b804b0bb556599a26d923ac, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:23.744232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582791b-52ff-4ed2-9095-e0ffb4cea07d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:23.793876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:23.825823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1035/2079] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:23 2022] PUT /identity/v3/projects/dad90b6c7dd947dda6faa1fc1feb6d7b/users/8d72e2cc5b804b0bb556599a26d923ac/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:23.831007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a9abe-9ac1-461e-9537-6bfc709362bd None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:23.831516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a9abe-9ac1-461e-9537-6bfc709362bd None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:23.831611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a9abe-9ac1-461e-9537-6bfc709362bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:23.852818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ee5a9abe-9ac1-461e-9537-6bfc709362bd None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:23.854097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ee5a9abe-9ac1-461e-9537-6bfc709362bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:23.920782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1045/2080] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:23.930974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582791b-52ff-4ed2-9095-e0ffb4cea07d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:23.932668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f582791b-52ff-4ed2-9095-e0ffb4cea07d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:24.003158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582791b-52ff-4ed2-9095-e0ffb4cea07d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:24.005612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:24.010823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f28b8f-2d5f-4bbb-8c1b-bf61275f78d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:24.011507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f28b8f-2d5f-4bbb-8c1b-bf61275f78d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:24.011779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f28b8f-2d5f-4bbb-8c1b-bf61275f78d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:24.012052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f28b8f-2d5f-4bbb-8c1b-bf61275f78d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:24.022004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f28b8f-2d5f-4bbb-8c1b-bf61275f78d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:24.023442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f28b8f-2d5f-4bbb-8c1b-bf61275f78d6 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:24.033594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1036/2081] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:24.035134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:24.212834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:24.246903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:24.521018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:24.636426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:25.281047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:25.360830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:26.309677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:26.343967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:27.007361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:28.953666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:30.132488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85661efc-8ae3-4808-ab82-d495e4c7fe11 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:30.133068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85661efc-8ae3-4808-ab82-d495e4c7fe11 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:30.133272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85661efc-8ae3-4808-ab82-d495e4c7fe11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:30.165540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-85661efc-8ae3-4808-ab82-d495e4c7fe11 None None] MFA Rules not processed for user `8d72e2cc5b804b0bb556599a26d923ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:30.166975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-85661efc-8ae3-4808-ab82-d495e4c7fe11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:30.254162 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1046/2082] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:30.277121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f28b8f-2d5f-4bbb-8c1b-bf61275f78d6 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:30.279032 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-38f28b8f-2d5f-4bbb-8c1b-bf61275f78d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:30.386225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f28b8f-2d5f-4bbb-8c1b-bf61275f78d6 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:30.389968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:30.395343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ed3fa-346a-4be2-898d-84143c8ea5cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:30.396001 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ed3fa-346a-4be2-898d-84143c8ea5cd service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:30.396253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ed3fa-346a-4be2-898d-84143c8ea5cd service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:30.396528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ed3fa-346a-4be2-898d-84143c8ea5cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:30.410566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ed3fa-346a-4be2-898d-84143c8ea5cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:30.416912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ed3fa-346a-4be2-898d-84143c8ea5cd service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:30.427932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1037/2083] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:31.097596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85661efc-8ae3-4808-ab82-d495e4c7fe11 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:31.099021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-85661efc-8ae3-4808-ab82-d495e4c7fe11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:31.116280 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85661efc-8ae3-4808-ab82-d495e4c7fe11 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:31.122219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:31.122879 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:31.123170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:31.123466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] PATH_INFO: `/v3/users/1d44d26266834112834c1ee9ae97f4d0` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:31.124741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d44d26266834112834c1ee9ae97f4d0)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:31.126744 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:31.384561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d44d26266834112834c1ee9ae97f4d0'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:31.385615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d44d26266834112834c1ee9ae97f4d0'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:31.395877 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d44d26266834112834c1ee9ae97f4d0'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:31.410639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d44d26266834112834c1ee9ae97f4d0'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:31.479690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d44d26266834112834c1ee9ae97f4d0'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:31.486887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1047/2084] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:31 2022] DELETE /identity/v3/users/1d44d26266834112834c1ee9ae97f4d0 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:31.492741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc72a-1bf3-43b2-b980-18975f917d32 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:31.493298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc72a-1bf3-43b2-b980-18975f917d32 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:31.493298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc72a-1bf3-43b2-b980-18975f917d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:31.531542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-06edc72a-1bf3-43b2-b980-18975f917d32 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:31.532757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-06edc72a-1bf3-43b2-b980-18975f917d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:31.558360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:31.628047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1038/2085] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:31.645357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:31.647178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:31.733687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4c386-30ee-4c56-9eed-14a7fa2d2655 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:31.739351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6efaf-84b8-4ff4-8aa2-610f60375060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:31.740423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6efaf-84b8-4ff4-8aa2-610f60375060 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:31.741721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6efaf-84b8-4ff4-8aa2-610f60375060 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:31.742455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6efaf-84b8-4ff4-8aa2-610f60375060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:31.757104 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6efaf-84b8-4ff4-8aa2-610f60375060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:31.758662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6efaf-84b8-4ff4-8aa2-610f60375060 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:31.771393 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1048/2086] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:32.331761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edc72a-1bf3-43b2-b980-18975f917d32 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:32.333210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-06edc72a-1bf3-43b2-b980-18975f917d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:32.352966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edc72a-1bf3-43b2-b980-18975f917d32 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:32.358081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:32.358722 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:32.359007 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:32.359304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] PATH_INFO: `/v3/projects/60b41069ee7a4e678ef544404221f65d` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:32.360656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b41069ee7a4e678ef544404221f65d)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:32.362587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:32.375874 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:32.512689 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b41069ee7a4e678ef544404221f65d', 'request_id': 'req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:32.520213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b41069ee7a4e678ef544404221f65d', 'request_id': 'req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:32.530744 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b41069ee7a4e678ef544404221f65d', 'request_id': 'req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:32.665715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] The token cache is being invalidate because project 60b41069ee7a4e678ef544404221f65d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:32.665715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:32.665715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-63e4cf77-19ba-4948-b04d-5d56ed8ed94b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:32.666262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1039/2087] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:32 2022] DELETE /identity/v3/projects/60b41069ee7a4e678ef544404221f65d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:32.671265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bb4f7-ec3a-426c-9a1a-449ea35d5731 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:32.671846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bb4f7-ec3a-426c-9a1a-449ea35d5731 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:32.671995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bb4f7-ec3a-426c-9a1a-449ea35d5731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:32.695649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-365bb4f7-ec3a-426c-9a1a-449ea35d5731 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:32.696910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-365bb4f7-ec3a-426c-9a1a-449ea35d5731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:32.713099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:32.775202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1049/2088] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:32.781205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821adfd-8d5e-401b-9adb-bc14af4bc2bc None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:32.781491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821adfd-8d5e-401b-9adb-bc14af4bc2bc None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:32.781773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821adfd-8d5e-401b-9adb-bc14af4bc2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:32.812773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-9821adfd-8d5e-401b-9adb-bc14af4bc2bc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:32.814354 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9821adfd-8d5e-401b-9adb-bc14af4bc2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:32.850395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1040/2089] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:32.854935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bb4f7-ec3a-426c-9a1a-449ea35d5731 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:32.894569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bb4f7-ec3a-426c-9a1a-449ea35d5731 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:32.918498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92dba6-d5aa-4f79-87a0-65eb49651ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:32.920695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92dba6-d5aa-4f79-87a0-65eb49651ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:32.925003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92dba6-d5aa-4f79-87a0-65eb49651ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:32.925633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92dba6-d5aa-4f79-87a0-65eb49651ce2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:32.926642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92dba6-d5aa-4f79-87a0-65eb49651ce2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:18:32.927125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92dba6-d5aa-4f79-87a0-65eb49651ce2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:32.929538 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92dba6-d5aa-4f79-87a0-65eb49651ce2 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:32.978926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1050/2090] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:18:32.990204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821adfd-8d5e-401b-9adb-bc14af4bc2bc None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:33.003696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821adfd-8d5e-401b-9adb-bc14af4bc2bc None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:33.011891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb06d42-c8c4-46d4-97a7-828e74e48999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:33.012684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06d42-c8c4-46d4-97a7-828e74e48999 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:33.013103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06d42-c8c4-46d4-97a7-828e74e48999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:33.013475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06d42-c8c4-46d4-97a7-828e74e48999 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:33.014226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb06d42-c8c4-46d4-97a7-828e74e48999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:33.016294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb06d42-c8c4-46d4-97a7-828e74e48999 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:33.149355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1041/2091] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:32 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:33.153353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92dba6-d5aa-4f79-87a0-65eb49651ce2 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:33.174357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92dba6-d5aa-4f79-87a0-65eb49651ce2 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:33.275602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6e494-3deb-4560-9f59-53e3cdcaffe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:33.276240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6e494-3deb-4560-9f59-53e3cdcaffe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:33.276524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6e494-3deb-4560-9f59-53e3cdcaffe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:33.276798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6e494-3deb-4560-9f59-53e3cdcaffe1 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:33.277522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6e494-3deb-4560-9f59-53e3cdcaffe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:33.279671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6e494-3deb-4560-9f59-53e3cdcaffe1 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:33.394870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482e360-5c6b-4cca-8522-e1705fa689e0 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:33.395496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482e360-5c6b-4cca-8522-e1705fa689e0 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:33.395953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482e360-5c6b-4cca-8522-e1705fa689e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:33.425108 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1051/2092] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:33 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:18:33.429962 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1482e360-5c6b-4cca-8522-e1705fa689e0 None None] MFA Rules not processed for user `8d72e2cc5b804b0bb556599a26d923ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:33.431481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1482e360-5c6b-4cca-8522-e1705fa689e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:33.431863 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6e494-3deb-4560-9f59-53e3cdcaffe1 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:33.446572 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6e494-3deb-4560-9f59-53e3cdcaffe1 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:33.455978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aaee1e-a3e5-4a12-9dd9-cf7c3c4e7f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:33.456661 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aaee1e-a3e5-4a12-9dd9-cf7c3c4e7f90 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:33.459198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aaee1e-a3e5-4a12-9dd9-cf7c3c4e7f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:33.459198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aaee1e-a3e5-4a12-9dd9-cf7c3c4e7f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:33.459198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aaee1e-a3e5-4a12-9dd9-cf7c3c4e7f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:33.462198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aaee1e-a3e5-4a12-9dd9-cf7c3c4e7f90 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:33.483615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1052/2093] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:33.487645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aaee1e-a3e5-4a12-9dd9-cf7c3c4e7f90 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:33.503159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aaee1e-a3e5-4a12-9dd9-cf7c3c4e7f90 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:33.509691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7268d-eecb-44a7-8e58-01d19b01367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:33.510546 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7268d-eecb-44a7-8e58-01d19b01367d admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:33.511182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7268d-eecb-44a7-8e58-01d19b01367d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:33.511629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7268d-eecb-44a7-8e58-01d19b01367d admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:33.512553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7268d-eecb-44a7-8e58-01d19b01367d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:33.514557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7268d-eecb-44a7-8e58-01d19b01367d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:33.516822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1042/2094] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:33.528033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1053/2095] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:33.532521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482e360-5c6b-4cca-8522-e1705fa689e0 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:33.545613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482e360-5c6b-4cca-8522-e1705fa689e0 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:33.555938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55747c95-2b0c-4c1a-a0d4-58abec4350d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:33.557601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55747c95-2b0c-4c1a-a0d4-58abec4350d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:33.558227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55747c95-2b0c-4c1a-a0d4-58abec4350d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:33.558713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55747c95-2b0c-4c1a-a0d4-58abec4350d7 admin admin] PATH_INFO: `/v3/projects/863d87bfa72b481f948bbd9cabc45d93/users/d071cf16a6bd4e3ead953f9de53aee35/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:33.584107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55747c95-2b0c-4c1a-a0d4-58abec4350d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=863d87bfa72b481f948bbd9cabc45d93, user_id=d071cf16a6bd4e3ead953f9de53aee35, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:33.587218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55747c95-2b0c-4c1a-a0d4-58abec4350d7 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:33.721260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1043/2096] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:33 2022] PUT /identity/v3/projects/863d87bfa72b481f948bbd9cabc45d93/users/d071cf16a6bd4e3ead953f9de53aee35/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:33.737651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaac189-e373-4061-a741-a764b8ef0fb9 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:33.738233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaac189-e373-4061-a741-a764b8ef0fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:33.738351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaac189-e373-4061-a741-a764b8ef0fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:33.776559 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-dbaac189-e373-4061-a741-a764b8ef0fb9 None None] MFA Rules not processed for user `d071cf16a6bd4e3ead953f9de53aee35`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:33.779713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-dbaac189-e373-4061-a741-a764b8ef0fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:33.880483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1054/2097] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:33.886105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55747c95-2b0c-4c1a-a0d4-58abec4350d7 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:33.950561 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:33.962876 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55747c95-2b0c-4c1a-a0d4-58abec4350d7 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:33.972584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3d25c-3a03-48e4-b49b-1605a85a5c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:33.974999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3d25c-3a03-48e4-b49b-1605a85a5c8d admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:33.975812 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3d25c-3a03-48e4-b49b-1605a85a5c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:33.976116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3d25c-3a03-48e4-b49b-1605a85a5c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:33.976894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3d25c-3a03-48e4-b49b-1605a85a5c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:33.978870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3d25c-3a03-48e4-b49b-1605a85a5c8d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:34.136397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1044/2098] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:33 2022] POST /identity/v3/projects => generated 383 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:18:34.138179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaac189-e373-4061-a741-a764b8ef0fb9 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:34.159714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaac189-e373-4061-a741-a764b8ef0fb9 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:34.237293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1766d-475c-4911-a4b0-95125cd504af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:34.238152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1766d-475c-4911-a4b0-95125cd504af admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:34.238360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1766d-475c-4911-a4b0-95125cd504af admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:34.238690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1766d-475c-4911-a4b0-95125cd504af admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:34.239289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1766d-475c-4911-a4b0-95125cd504af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:34.242336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1766d-475c-4911-a4b0-95125cd504af admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:34.372176 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1055/2099] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:34 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:34.376173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3d25c-3a03-48e4-b49b-1605a85a5c8d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:34.388652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3d25c-3a03-48e4-b49b-1605a85a5c8d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:34.394091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed5207-8e1e-47c6-98ae-00945bc70b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:34.395078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed5207-8e1e-47c6-98ae-00945bc70b61 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:34.395250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed5207-8e1e-47c6-98ae-00945bc70b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:34.395566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed5207-8e1e-47c6-98ae-00945bc70b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:34.396138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed5207-8e1e-47c6-98ae-00945bc70b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:34.397918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:34.398671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed5207-8e1e-47c6-98ae-00945bc70b61 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:34.415861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1045/2100] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:34.421950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1766d-475c-4911-a4b0-95125cd504af admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:34.439244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1766d-475c-4911-a4b0-95125cd504af admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:34.451030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db644bc2-3559-43e0-a480-f8c84ab0caee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:34.451852 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db644bc2-3559-43e0-a480-f8c84ab0caee admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:34.452135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db644bc2-3559-43e0-a480-f8c84ab0caee admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:34.452410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db644bc2-3559-43e0-a480-f8c84ab0caee admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:34.453113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db644bc2-3559-43e0-a480-f8c84ab0caee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:34.454935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db644bc2-3559-43e0-a480-f8c84ab0caee admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:34.474658 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1056/2101] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:34.478520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed5207-8e1e-47c6-98ae-00945bc70b61 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:34.515028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed5207-8e1e-47c6-98ae-00945bc70b61 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:34.521755 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f2cd0-a51c-4113-82bb-c863407680c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:34.523422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f2cd0-a51c-4113-82bb-c863407680c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:34.523896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f2cd0-a51c-4113-82bb-c863407680c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:34.524402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f2cd0-a51c-4113-82bb-c863407680c7 admin admin] PATH_INFO: `/v3/projects/ceb9fa95756846aeb74ad8988572ef2b/users/1aab26d90b564153bd78f41a23bebae9/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:34.553741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f2cd0-a51c-4113-82bb-c863407680c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb9fa95756846aeb74ad8988572ef2b, user_id=1aab26d90b564153bd78f41a23bebae9, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:34.562434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f2cd0-a51c-4113-82bb-c863407680c7 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:34.707073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1046/2102] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:34 2022] PUT /identity/v3/projects/ceb9fa95756846aeb74ad8988572ef2b/users/1aab26d90b564153bd78f41a23bebae9/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:34.709510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107522a7-0b94-462a-becb-714da888f4a5 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:34.710426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107522a7-0b94-462a-becb-714da888f4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:34.710812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107522a7-0b94-462a-becb-714da888f4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:34.740746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-107522a7-0b94-462a-becb-714da888f4a5 None None] MFA Rules not processed for user `1aab26d90b564153bd78f41a23bebae9`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:34.742139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-107522a7-0b94-462a-becb-714da888f4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:34.813535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1057/2103] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:34.818360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f2cd0-a51c-4113-82bb-c863407680c7 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:34.902476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f2cd0-a51c-4113-82bb-c863407680c7 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:34.908291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262bf6-4a3a-45a7-bc07-881dcb3edb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:34.909130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262bf6-4a3a-45a7-bc07-881dcb3edb92 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:34.909839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262bf6-4a3a-45a7-bc07-881dcb3edb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:34.910127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262bf6-4a3a-45a7-bc07-881dcb3edb92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:34.910934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d262bf6-4a3a-45a7-bc07-881dcb3edb92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:34.913401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d262bf6-4a3a-45a7-bc07-881dcb3edb92 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:35.083015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1047/2104] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:34 2022] POST /identity/v3/projects => generated 383 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:35.087931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107522a7-0b94-462a-becb-714da888f4a5 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:35.100818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107522a7-0b94-462a-becb-714da888f4a5 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:35.160499 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096388f-9203-4495-ba92-7a195ba52639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:35.162330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096388f-9203-4495-ba92-7a195ba52639 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:35.162330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096388f-9203-4495-ba92-7a195ba52639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:35.162330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096388f-9203-4495-ba92-7a195ba52639 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:35.162330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096388f-9203-4495-ba92-7a195ba52639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:35.164763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096388f-9203-4495-ba92-7a195ba52639 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:35.327926 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1058/2105] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:35 2022] POST /identity/v3/users => generated 298 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:35.331929 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262bf6-4a3a-45a7-bc07-881dcb3edb92 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:35.342671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262bf6-4a3a-45a7-bc07-881dcb3edb92 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:35.347223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af7bc6-d6c3-499d-8710-4b9f1acec353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:35.347824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af7bc6-d6c3-499d-8710-4b9f1acec353 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:35.348105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af7bc6-d6c3-499d-8710-4b9f1acec353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:35.350732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af7bc6-d6c3-499d-8710-4b9f1acec353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:35.350732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af7bc6-d6c3-499d-8710-4b9f1acec353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:35.350927 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af7bc6-d6c3-499d-8710-4b9f1acec353 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:35.365513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1048/2106] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:35.367584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096388f-9203-4495-ba92-7a195ba52639 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:35.377588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096388f-9203-4495-ba92-7a195ba52639 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:35.383103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c312c-4039-4dd6-a64b-5e453292ecc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:35.383754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c312c-4039-4dd6-a64b-5e453292ecc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:35.384037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c312c-4039-4dd6-a64b-5e453292ecc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:35.384312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c312c-4039-4dd6-a64b-5e453292ecc7 admin admin] PATH_INFO: `/v3/projects/0b91db6b85c543b38be6c109874d944b/users/1210eb26b9154c8ca43ef8e2f086e70d/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:35.399146 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c312c-4039-4dd6-a64b-5e453292ecc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b91db6b85c543b38be6c109874d944b, user_id=1210eb26b9154c8ca43ef8e2f086e70d, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:35.401860 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c312c-4039-4dd6-a64b-5e453292ecc7 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:35.457928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:35.534651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1059/2107] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:35 2022] PUT /identity/v3/projects/0b91db6b85c543b38be6c109874d944b/users/1210eb26b9154c8ca43ef8e2f086e70d/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:35.539357 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af7bc6-d6c3-499d-8710-4b9f1acec353 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:35.559013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af7bc6-d6c3-499d-8710-4b9f1acec353 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:35.619387 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97728bdd-195d-4056-b3d3-b5e7f81b136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:35.620003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97728bdd-195d-4056-b3d3-b5e7f81b136c admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:35.620299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97728bdd-195d-4056-b3d3-b5e7f81b136c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:35.620592 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97728bdd-195d-4056-b3d3-b5e7f81b136c admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:35.621450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97728bdd-195d-4056-b3d3-b5e7f81b136c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:35.623054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97728bdd-195d-4056-b3d3-b5e7f81b136c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:35.633551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1049/2108] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:35.637990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c312c-4039-4dd6-a64b-5e453292ecc7 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:35.649381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c312c-4039-4dd6-a64b-5e453292ecc7 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:35.654190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cf800-3a64-43dc-9477-b3230f33106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:35.654857 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cf800-3a64-43dc-9477-b3230f33106d admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:35.655145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cf800-3a64-43dc-9477-b3230f33106d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:35.655446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cf800-3a64-43dc-9477-b3230f33106d admin admin] PATH_INFO: `/v3/projects/0b91db6b85c543b38be6c109874d944b/users/1210eb26b9154c8ca43ef8e2f086e70d/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:35.657630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cf800-3a64-43dc-9477-b3230f33106d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b91db6b85c543b38be6c109874d944b, user_id=1210eb26b9154c8ca43ef8e2f086e70d, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:35.660284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cf800-3a64-43dc-9477-b3230f33106d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:35.671372 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1060/2109] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:35 2022] PUT /identity/v3/projects/0b91db6b85c543b38be6c109874d944b/users/1210eb26b9154c8ca43ef8e2f086e70d/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:35.682292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a3e7a-a536-470e-b09e-ef2ef4c3fc80 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:35.682581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a3e7a-a536-470e-b09e-ef2ef4c3fc80 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:35.682860 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a3e7a-a536-470e-b09e-ef2ef4c3fc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:35.723448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:35.727439 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-f11a3e7a-a536-470e-b09e-ef2ef4c3fc80 None None] MFA Rules not processed for user `1210eb26b9154c8ca43ef8e2f086e70d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:35.729278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f11a3e7a-a536-470e-b09e-ef2ef4c3fc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:35.813725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1050/2110] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:35.826558 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cf800-3a64-43dc-9477-b3230f33106d admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:35.828386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6f9cf800-3a64-43dc-9477-b3230f33106d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:35.889008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cf800-3a64-43dc-9477-b3230f33106d admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:35.892649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64cc41-885e-4cdc-b74c-3b7979b813bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:35.893292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64cc41-885e-4cdc-b74c-3b7979b813bb service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:35.893568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64cc41-885e-4cdc-b74c-3b7979b813bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:35.893907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64cc41-885e-4cdc-b74c-3b7979b813bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:35.950506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64cc41-885e-4cdc-b74c-3b7979b813bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:35.951945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64cc41-885e-4cdc-b74c-3b7979b813bb service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:35.962010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1061/2111] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:39.127366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a3e7a-a536-470e-b09e-ef2ef4c3fc80 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:39.137602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a3e7a-a536-470e-b09e-ef2ef4c3fc80 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:39.141158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0869e2e-906d-4d60-9071-ac2cce4e0f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:39.141754 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0869e2e-906d-4d60-9071-ac2cce4e0f3c service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:39.142036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0869e2e-906d-4d60-9071-ac2cce4e0f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:39.142311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0869e2e-906d-4d60-9071-ac2cce4e0f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:39.198353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0869e2e-906d-4d60-9071-ac2cce4e0f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:39.199779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0869e2e-906d-4d60-9071-ac2cce4e0f3c service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:39.208576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1051/2112] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:41.849875 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64cc41-885e-4cdc-b74c-3b7979b813bb service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:41.869169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64cc41-885e-4cdc-b74c-3b7979b813bb service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:41.869169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0711a-de8f-4a16-b224-f1b1f62066c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:41.869169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0711a-de8f-4a16-b224-f1b1f62066c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:41.869169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0711a-de8f-4a16-b224-f1b1f62066c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:41.869535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0711a-de8f-4a16-b224-f1b1f62066c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:41.880533 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0711a-de8f-4a16-b224-f1b1f62066c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:41.882496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0711a-de8f-4a16-b224-f1b1f62066c3 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:41.895206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1062/2113] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:45.970642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0869e2e-906d-4d60-9071-ac2cce4e0f3c service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:45.972112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a0869e2e-906d-4d60-9071-ac2cce4e0f3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:46.040566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0869e2e-906d-4d60-9071-ac2cce4e0f3c service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:46.046329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:46.047449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:46.047449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:46.047623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] PATH_INFO: `/v3/users/45c693ce56dd43f1be05223c81d8c49c` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:46.048689 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c693ce56dd43f1be05223c81d8c49c)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:46.050699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:46.271666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:46.278368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c693ce56dd43f1be05223c81d8c49c'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:46.279042 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c693ce56dd43f1be05223c81d8c49c'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:46.286595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c693ce56dd43f1be05223c81d8c49c'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:46.287290 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c693ce56dd43f1be05223c81d8c49c'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:46.383078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c693ce56dd43f1be05223c81d8c49c'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:46.404096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1052/2114] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:45 2022] DELETE /identity/v3/users/45c693ce56dd43f1be05223c81d8c49c => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:46.408662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990497e-8a1d-499b-bbc0-f626538bc572 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:46.409184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990497e-8a1d-499b-bbc0-f626538bc572 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:46.409298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990497e-8a1d-499b-bbc0-f626538bc572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:46.429998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-5990497e-8a1d-499b-bbc0-f626538bc572 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:46.431218 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5990497e-8a1d-499b-bbc0-f626538bc572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:46.498786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1063/2115] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:46.508516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:46.510088 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:46.586012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553439ed-0731-439c-82d4-0e6ff456321d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:46.590807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b25ef-36ab-4d69-a2a2-de77ab33318c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:46.591501 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b25ef-36ab-4d69-a2a2-de77ab33318c service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:46.591784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b25ef-36ab-4d69-a2a2-de77ab33318c service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:46.592064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b25ef-36ab-4d69-a2a2-de77ab33318c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:46.601861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b25ef-36ab-4d69-a2a2-de77ab33318c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:46.603865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b25ef-36ab-4d69-a2a2-de77ab33318c service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:46.614849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1053/2116] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:46.986398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990497e-8a1d-499b-bbc0-f626538bc572 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:46.987992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5990497e-8a1d-499b-bbc0-f626538bc572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:46.997100 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:47.000819 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990497e-8a1d-499b-bbc0-f626538bc572 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.005743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81737ac3-f3f4-4dec-beb9-1c0dfc11da60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.006365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81737ac3-f3f4-4dec-beb9-1c0dfc11da60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.006646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81737ac3-f3f4-4dec-beb9-1c0dfc11da60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.006923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81737ac3-f3f4-4dec-beb9-1c0dfc11da60 admin admin] PATH_INFO: `/v3/users/d071cf16a6bd4e3ead953f9de53aee35` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.008169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81737ac3-f3f4-4dec-beb9-1c0dfc11da60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d071cf16a6bd4e3ead953f9de53aee35)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.010152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81737ac3-f3f4-4dec-beb9-1c0dfc11da60 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.047878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b25ef-36ab-4d69-a2a2-de77ab33318c service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.049356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a60b25ef-36ab-4d69-a2a2-de77ab33318c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:47.063014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b25ef-36ab-4d69-a2a2-de77ab33318c service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.068781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.069479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.069758 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.070034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] PATH_INFO: `/v3/projects/4040505dee3440ecb273c5d4fb07a0d2` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.071162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4040505dee3440ecb273c5d4fb07a0d2)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.077873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.192081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-81737ac3-f3f4-4dec-beb9-1c0dfc11da60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd071cf16a6bd4e3ead953f9de53aee35'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.192736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-81737ac3-f3f4-4dec-beb9-1c0dfc11da60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd071cf16a6bd4e3ead953f9de53aee35'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.201888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-81737ac3-f3f4-4dec-beb9-1c0dfc11da60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd071cf16a6bd4e3ead953f9de53aee35'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.214808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-81737ac3-f3f4-4dec-beb9-1c0dfc11da60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd071cf16a6bd4e3ead953f9de53aee35'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.225562 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4040505dee3440ecb273c5d4fb07a0d2', 'request_id': 'req-0dc548c6-2338-45e8-b939-adffcb7da509'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.233056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4040505dee3440ecb273c5d4fb07a0d2', 'request_id': 'req-0dc548c6-2338-45e8-b939-adffcb7da509'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.239923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4040505dee3440ecb273c5d4fb07a0d2', 'request_id': 'req-0dc548c6-2338-45e8-b939-adffcb7da509'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.331802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-81737ac3-f3f4-4dec-beb9-1c0dfc11da60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd071cf16a6bd4e3ead953f9de53aee35'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.338381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1064/2117] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:46 2022] DELETE /identity/v3/users/d071cf16a6bd4e3ead953f9de53aee35 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:47.342871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e8d49-2520-4240-b5e4-4d6399c7541d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.343159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e8d49-2520-4240-b5e4-4d6399c7541d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.343491 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e8d49-2520-4240-b5e4-4d6399c7541d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.363574 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-f43e8d49-2520-4240-b5e4-4d6399c7541d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:47.365185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f43e8d49-2520-4240-b5e4-4d6399c7541d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:47.399939 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] The token cache is being invalidate because project 4040505dee3440ecb273c5d4fb07a0d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:47.400234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.400952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0dc548c6-2338-45e8-b939-adffcb7da509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.402623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1054/2118] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:47 2022] DELETE /identity/v3/projects/4040505dee3440ecb273c5d4fb07a0d2 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:47.407287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4fca0-153d-48c9-b60b-d51929690b19 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.407573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4fca0-153d-48c9-b60b-d51929690b19 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.407875 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4fca0-153d-48c9-b60b-d51929690b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.428924 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-86e4fca0-153d-48c9-b60b-d51929690b19 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:47.429890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1065/2119] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:47.430344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-86e4fca0-153d-48c9-b60b-d51929690b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:47.441548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e8d49-2520-4240-b5e4-4d6399c7541d None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.442898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f43e8d49-2520-4240-b5e4-4d6399c7541d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:47.447062 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1055/2120] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:47.455940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522dbf0-b02d-482f-b948-c644dbcfc4cb None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.456433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522dbf0-b02d-482f-b948-c644dbcfc4cb None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.456906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522dbf0-b02d-482f-b948-c644dbcfc4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.477405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6522dbf0-b02d-482f-b948-c644dbcfc4cb None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:47.478606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6522dbf0-b02d-482f-b948-c644dbcfc4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:47.487383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1056/2121] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:47.495523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522dbf0-b02d-482f-b948-c644dbcfc4cb None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.506319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522dbf0-b02d-482f-b948-c644dbcfc4cb None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.513539 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155903a1-62a7-4e9c-8fb2-5cf8c5e50d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.514169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155903a1-62a7-4e9c-8fb2-5cf8c5e50d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.514451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155903a1-62a7-4e9c-8fb2-5cf8c5e50d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.514727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155903a1-62a7-4e9c-8fb2-5cf8c5e50d62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.515390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155903a1-62a7-4e9c-8fb2-5cf8c5e50d62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:18:47.515702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155903a1-62a7-4e9c-8fb2-5cf8c5e50d62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.517346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155903a1-62a7-4e9c-8fb2-5cf8c5e50d62 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.521007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e8d49-2520-4240-b5e4-4d6399c7541d None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.525413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d317f78-c103-4ca7-bcf4-0f912a94e96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.525562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d317f78-c103-4ca7-bcf4-0f912a94e96e service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.525849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d317f78-c103-4ca7-bcf4-0f912a94e96e service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.526163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d317f78-c103-4ca7-bcf4-0f912a94e96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.532908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1057/2122] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:18:47.536888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d317f78-c103-4ca7-bcf4-0f912a94e96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.537191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155903a1-62a7-4e9c-8fb2-5cf8c5e50d62 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.538544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d317f78-c103-4ca7-bcf4-0f912a94e96e service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.548555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155903a1-62a7-4e9c-8fb2-5cf8c5e50d62 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.550704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1066/2123] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:18:47.553643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3588-8bb5-4bf7-9c01-70605319a7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.554249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3588-8bb5-4bf7-9c01-70605319a7ce admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.554532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3588-8bb5-4bf7-9c01-70605319a7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.554810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3588-8bb5-4bf7-9c01-70605319a7ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.555469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a3588-8bb5-4bf7-9c01-70605319a7ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.558059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a3588-8bb5-4bf7-9c01-70605319a7ce admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.574623 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:47.722803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1058/2124] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:47 2022] POST /identity/v3/projects => generated 361 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:47.727370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d317f78-c103-4ca7-bcf4-0f912a94e96e service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.738289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d317f78-c103-4ca7-bcf4-0f912a94e96e service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.756741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:47.797984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741e929-fd21-4fbb-8cf4-5b438e6bc12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.798597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741e929-fd21-4fbb-8cf4-5b438e6bc12c admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.799076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741e929-fd21-4fbb-8cf4-5b438e6bc12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.799298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741e929-fd21-4fbb-8cf4-5b438e6bc12c admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.799988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741e929-fd21-4fbb-8cf4-5b438e6bc12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.802650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741e929-fd21-4fbb-8cf4-5b438e6bc12c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.878226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3588-8bb5-4bf7-9c01-70605319a7ce admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.880278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-994a3588-8bb5-4bf7-9c01-70605319a7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:47.894031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3588-8bb5-4bf7-9c01-70605319a7ce admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.908114 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64745256-72fa-4ba3-a661-bb6a864e5bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.908430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745256-72fa-4ba3-a661-bb6a864e5bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.908842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745256-72fa-4ba3-a661-bb6a864e5bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.909304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745256-72fa-4ba3-a661-bb6a864e5bec admin admin] PATH_INFO: `/v3/users/beff0559f8c0471aa4a6496f02576442` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.910517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64745256-72fa-4ba3-a661-bb6a864e5bec admin admin] RBAC: Authorizing `identity:delete_user(user_id=beff0559f8c0471aa4a6496f02576442)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.912513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64745256-72fa-4ba3-a661-bb6a864e5bec admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.939145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1067/2125] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:47 2022] POST /identity/v3/users => generated 281 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:47.942847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741e929-fd21-4fbb-8cf4-5b438e6bc12c admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.953655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741e929-fd21-4fbb-8cf4-5b438e6bc12c admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.958873 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1c7af-5700-4a5c-916e-12e00ac7bde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.966178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c7af-5700-4a5c-916e-12e00ac7bde5 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.966178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c7af-5700-4a5c-916e-12e00ac7bde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.966178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c7af-5700-4a5c-916e-12e00ac7bde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.966178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1c7af-5700-4a5c-916e-12e00ac7bde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.966178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1c7af-5700-4a5c-916e-12e00ac7bde5 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.974064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1068/2126] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:18:47.979187 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1c7af-5700-4a5c-916e-12e00ac7bde5 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.989913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1c7af-5700-4a5c-916e-12e00ac7bde5 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.994728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f94e25-2ee8-486e-86dc-2c750a1ee16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.995378 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f94e25-2ee8-486e-86dc-2c750a1ee16b admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.995727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f94e25-2ee8-486e-86dc-2c750a1ee16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.996126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f94e25-2ee8-486e-86dc-2c750a1ee16b admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.996777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f94e25-2ee8-486e-86dc-2c750a1ee16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.998411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f94e25-2ee8-486e-86dc-2c750a1ee16b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:48.009616 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1069/2127] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:48.014244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f94e25-2ee8-486e-86dc-2c750a1ee16b admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:48.024403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f94e25-2ee8-486e-86dc-2c750a1ee16b admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:48.030415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa0438-cf7e-4429-bd42-2b1d47868c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:48.031060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa0438-cf7e-4429-bd42-2b1d47868c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.031337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa0438-cf7e-4429-bd42-2b1d47868c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.031609 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa0438-cf7e-4429-bd42-2b1d47868c1f admin admin] PATH_INFO: `/v3/projects/aea86f445aad428d896a1fc0c71c6fef/users/d58425b7779c491a90895f917a200b44/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.046763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa0438-cf7e-4429-bd42-2b1d47868c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea86f445aad428d896a1fc0c71c6fef, user_id=d58425b7779c491a90895f917a200b44, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:48.049601 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa0438-cf7e-4429-bd42-2b1d47868c1f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:48.085362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-64745256-72fa-4ba3-a661-bb6a864e5bec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beff0559f8c0471aa4a6496f02576442'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.086037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-64745256-72fa-4ba3-a661-bb6a864e5bec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beff0559f8c0471aa4a6496f02576442'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.098821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-64745256-72fa-4ba3-a661-bb6a864e5bec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beff0559f8c0471aa4a6496f02576442'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.098821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-64745256-72fa-4ba3-a661-bb6a864e5bec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beff0559f8c0471aa4a6496f02576442'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.143754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1070/2128] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:48 2022] PUT /identity/v3/projects/aea86f445aad428d896a1fc0c71c6fef/users/d58425b7779c491a90895f917a200b44/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:48.144733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa0438-cf7e-4429-bd42-2b1d47868c1f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:48.146431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7aaa0438-cf7e-4429-bd42-2b1d47868c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:48.194554 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-64745256-72fa-4ba3-a661-bb6a864e5bec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beff0559f8c0471aa4a6496f02576442'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.212946 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa0438-cf7e-4429-bd42-2b1d47868c1f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:48.214892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1059/2129] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:47 2022] DELETE /identity/v3/users/beff0559f8c0471aa4a6496f02576442 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:48.216406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3ddf8-9a4a-4ef4-9638-2555b8223fba None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.216687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3ddf8-9a4a-4ef4-9638-2555b8223fba None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.216979 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3ddf8-9a4a-4ef4-9638-2555b8223fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.247350 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-14e3ddf8-9a4a-4ef4-9638-2555b8223fba None None] MFA Rules not processed for user `d58425b7779c491a90895f917a200b44`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:48.249066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-14e3ddf8-9a4a-4ef4-9638-2555b8223fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:48.273856 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:48.274531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.274812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.275139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] PATH_INFO: `/v3/projects/863d87bfa72b481f948bbd9cabc45d93` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.277166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=863d87bfa72b481f948bbd9cabc45d93)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:48.279239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:48.319146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1060/2130] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:48.320566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcce5a-4cba-4044-8eac-8033a389ebbb None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.320909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcce5a-4cba-4044-8eac-8033a389ebbb None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.321163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcce5a-4cba-4044-8eac-8033a389ebbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.342378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e2bcce5a-4cba-4044-8eac-8033a389ebbb None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:48.342851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e2bcce5a-4cba-4044-8eac-8033a389ebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:48.350727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1061/2131] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:48.354388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcce5a-4cba-4044-8eac-8033a389ebbb None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:48.355870 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e2bcce5a-4cba-4044-8eac-8033a389ebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:48.377742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '863d87bfa72b481f948bbd9cabc45d93', 'request_id': 'req-a1ea142e-531d-4929-8bee-418d44127862'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.383366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '863d87bfa72b481f948bbd9cabc45d93', 'request_id': 'req-a1ea142e-531d-4929-8bee-418d44127862'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.412222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:48.416039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcce5a-4cba-4044-8eac-8033a389ebbb None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:48.466511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d0465-4a4d-44c2-8341-4f030d356e5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:48.467125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d0465-4a4d-44c2-8341-4f030d356e5a service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.467405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d0465-4a4d-44c2-8341-4f030d356e5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.467683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d0465-4a4d-44c2-8341-4f030d356e5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.491521 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '863d87bfa72b481f948bbd9cabc45d93', 'request_id': 'req-a1ea142e-531d-4929-8bee-418d44127862'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.504330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] The token cache is being invalidate because project 863d87bfa72b481f948bbd9cabc45d93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:48.504545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.505241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.506754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1071/2132] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:48 2022] DELETE /identity/v3/projects/863d87bfa72b481f948bbd9cabc45d93 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:48.509986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:48.509986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:48.527208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d0465-4a4d-44c2-8341-4f030d356e5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:48.528705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d0465-4a4d-44c2-8341-4f030d356e5a service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:48.538504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1062/2133] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:18:48.539327 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d0465-4a4d-44c2-8341-4f030d356e5a service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:48.568310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea142e-531d-4929-8bee-418d44127862 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:48.572111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf16895-e246-495e-9b1d-7b35dce440b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:48.572723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf16895-e246-495e-9b1d-7b35dce440b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.573911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf16895-e246-495e-9b1d-7b35dce440b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.573911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf16895-e246-495e-9b1d-7b35dce440b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.575812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-fdf16895-e246-495e-9b1d-7b35dce440b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:48.602298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:48.615975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d0465-4a4d-44c2-8341-4f030d356e5a service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:48.621897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b21f7-a44f-42e3-a7ff-6820e2eed32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:48.622616 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b21f7-a44f-42e3-a7ff-6820e2eed32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.622905 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b21f7-a44f-42e3-a7ff-6820e2eed32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.623180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b21f7-a44f-42e3-a7ff-6820e2eed32e admin admin] PATH_INFO: `/v3/users/1aab26d90b564153bd78f41a23bebae9` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.624817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b21f7-a44f-42e3-a7ff-6820e2eed32e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aab26d90b564153bd78f41a23bebae9)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:48.626989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b21f7-a44f-42e3-a7ff-6820e2eed32e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:48.654727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf16895-e246-495e-9b1d-7b35dce440b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:48.656391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf16895-e246-495e-9b1d-7b35dce440b2 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:48.671401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1072/2134] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:48.672549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf16895-e246-495e-9b1d-7b35dce440b2 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:48.674232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-fdf16895-e246-495e-9b1d-7b35dce440b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:48.686741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf16895-e246-495e-9b1d-7b35dce440b2 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:48.692169 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:48.692883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.693168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.693590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] PATH_INFO: `/v3/users/d58425b7779c491a90895f917a200b44` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.694854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d58425b7779c491a90895f917a200b44)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:48.697186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:48.851164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-367b21f7-a44f-42e3-a7ff-6820e2eed32e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aab26d90b564153bd78f41a23bebae9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.851869 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-367b21f7-a44f-42e3-a7ff-6820e2eed32e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aab26d90b564153bd78f41a23bebae9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.862243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-367b21f7-a44f-42e3-a7ff-6820e2eed32e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aab26d90b564153bd78f41a23bebae9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.863884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-367b21f7-a44f-42e3-a7ff-6820e2eed32e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aab26d90b564153bd78f41a23bebae9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.922199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd58425b7779c491a90895f917a200b44'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.923164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd58425b7779c491a90895f917a200b44'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.934415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd58425b7779c491a90895f917a200b44'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.938096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-367b21f7-a44f-42e3-a7ff-6820e2eed32e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aab26d90b564153bd78f41a23bebae9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.969607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd58425b7779c491a90895f917a200b44'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:48.984400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1063/2135] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:48 2022] DELETE /identity/v3/users/1aab26d90b564153bd78f41a23bebae9 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:48.996061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:49.029131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd58425b7779c491a90895f917a200b44'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:49.036762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1073/2136] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:48 2022] DELETE /identity/v3/users/d58425b7779c491a90895f917a200b44 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:49.040992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:49.043874 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9e9bf-832f-48ed-a59c-676a8a781c88 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:49.044340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9e9bf-832f-48ed-a59c-676a8a781c88 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:49.044818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9e9bf-832f-48ed-a59c-676a8a781c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:49.071116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6df9e9bf-832f-48ed-a59c-676a8a781c88 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:49.072494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6df9e9bf-832f-48ed-a59c-676a8a781c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:49.137152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1064/2137] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:49.147269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:49.149888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:49.215037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fac9d-6252-4b70-b5dd-10f6f6fc41e2 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:49.218967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c264e-34ab-4a35-ac6d-30cf2c7635ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:49.219678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c264e-34ab-4a35-ac6d-30cf2c7635ae service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:49.220500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c264e-34ab-4a35-ac6d-30cf2c7635ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:49.220847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c264e-34ab-4a35-ac6d-30cf2c7635ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:49.235991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c264e-34ab-4a35-ac6d-30cf2c7635ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:49.238030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c264e-34ab-4a35-ac6d-30cf2c7635ae service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:49.268621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1074/2138] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:49.362884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9e9bf-832f-48ed-a59c-676a8a781c88 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:49.364390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6df9e9bf-832f-48ed-a59c-676a8a781c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:49.381723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9e9bf-832f-48ed-a59c-676a8a781c88 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:49.388409 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:49.390096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:49.393531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:49.393531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] PATH_INFO: `/v3/projects/c853841b06da473094805c56c4281174` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:49.393531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c853841b06da473094805c56c4281174)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:49.395428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:49.526856 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c853841b06da473094805c56c4281174', 'request_id': 'req-fc64d597-717e-4bc6-89b0-33e05db9f67a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:49.532307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c853841b06da473094805c56c4281174', 'request_id': 'req-fc64d597-717e-4bc6-89b0-33e05db9f67a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:49.538794 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c853841b06da473094805c56c4281174', 'request_id': 'req-fc64d597-717e-4bc6-89b0-33e05db9f67a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:49.556496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c264e-34ab-4a35-ac6d-30cf2c7635ae service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:49.558055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-329c264e-34ab-4a35-ac6d-30cf2c7635ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:49.570985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c264e-34ab-4a35-ac6d-30cf2c7635ae service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:49.612475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] The token cache is being invalidate because project c853841b06da473094805c56c4281174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:49.617517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:49.617517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:49.617517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1065/2139] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:49 2022] DELETE /identity/v3/projects/c853841b06da473094805c56c4281174 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:49.648417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:49.649369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:49.649487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:49.649801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] PATH_INFO: `/v3/projects/ceb9fa95756846aeb74ad8988572ef2b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:49.654192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb9fa95756846aeb74ad8988572ef2b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:49.654687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:49.774581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb9fa95756846aeb74ad8988572ef2b', 'request_id': 'req-5550d8da-f759-487f-bfc4-75bfc35930c6'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:49.780610 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb9fa95756846aeb74ad8988572ef2b', 'request_id': 'req-5550d8da-f759-487f-bfc4-75bfc35930c6'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:49.847895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:49.849476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:49.863302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64d597-717e-4bc6-89b0-33e05db9f67a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:49.872995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb9fa95756846aeb74ad8988572ef2b', 'request_id': 'req-5550d8da-f759-487f-bfc4-75bfc35930c6'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:49.884999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] The token cache is being invalidate because project ceb9fa95756846aeb74ad8988572ef2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:49.885504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:49.886330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:49.887844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1075/2140] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:49 2022] DELETE /identity/v3/projects/ceb9fa95756846aeb74ad8988572ef2b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:49.891588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:49.893097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:49.932046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:49.932046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:49.932046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:49.932046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] PATH_INFO: `/v3/projects/aea86f445aad428d896a1fc0c71c6fef` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:49.933444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea86f445aad428d896a1fc0c71c6fef)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:49.935349 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:49.965692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550d8da-f759-487f-bfc4-75bfc35930c6 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:49.971045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:49.971831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:49.972224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:49.972643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] PATH_INFO: `/v3/users/1210eb26b9154c8ca43ef8e2f086e70d` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:49.973818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1210eb26b9154c8ca43ef8e2f086e70d)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:49.975799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:49.980395 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:50.046709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea86f445aad428d896a1fc0c71c6fef', 'request_id': 'req-04d7c148-df27-43ed-8348-399290ee908d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:50.052309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea86f445aad428d896a1fc0c71c6fef', 'request_id': 'req-04d7c148-df27-43ed-8348-399290ee908d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:50.059278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea86f445aad428d896a1fc0c71c6fef', 'request_id': 'req-04d7c148-df27-43ed-8348-399290ee908d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:50.103159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:50.176647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] The token cache is being invalidate because project aea86f445aad428d896a1fc0c71c6fef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:50.177310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:50.177674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04d7c148-df27-43ed-8348-399290ee908d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:50.179250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1066/2141] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:49 2022] DELETE /identity/v3/projects/aea86f445aad428d896a1fc0c71c6fef => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:50.184602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed84b28-b0c6-406c-951b-549ddcf57795 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:50.184919 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed84b28-b0c6-406c-951b-549ddcf57795 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:50.185209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed84b28-b0c6-406c-951b-549ddcf57795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:50.205444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-0ed84b28-b0c6-406c-951b-549ddcf57795 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:50.206648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0ed84b28-b0c6-406c-951b-549ddcf57795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:50.249327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1210eb26b9154c8ca43ef8e2f086e70d'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:50.250042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1210eb26b9154c8ca43ef8e2f086e70d'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:50.257799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1210eb26b9154c8ca43ef8e2f086e70d'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:50.271160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1067/2142] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:50.278024 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1210eb26b9154c8ca43ef8e2f086e70d'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:50.279850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb8c09-8b69-46a1-af8d-f519fa30d207 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:50.280175 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb8c09-8b69-46a1-af8d-f519fa30d207 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:50.280456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb8c09-8b69-46a1-af8d-f519fa30d207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:50.304549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-50fb8c09-8b69-46a1-af8d-f519fa30d207 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:50.305823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-50fb8c09-8b69-46a1-af8d-f519fa30d207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:50.314475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1068/2143] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:50.318941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb8c09-8b69-46a1-af8d-f519fa30d207 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:50.329831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb8c09-8b69-46a1-af8d-f519fa30d207 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:50.332715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1210eb26b9154c8ca43ef8e2f086e70d'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:50.336218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83886a32-ca11-49f0-b99f-fef61ddcd944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:50.337056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83886a32-ca11-49f0-b99f-fef61ddcd944 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:50.337249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83886a32-ca11-49f0-b99f-fef61ddcd944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:50.337501 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83886a32-ca11-49f0-b99f-fef61ddcd944 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:50.338160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83886a32-ca11-49f0-b99f-fef61ddcd944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:18:50.338555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83886a32-ca11-49f0-b99f-fef61ddcd944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:50.340391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83886a32-ca11-49f0-b99f-fef61ddcd944 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:50.343944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1076/2144] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:49 2022] DELETE /identity/v3/users/1210eb26b9154c8ca43ef8e2f086e70d => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:50.355487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1069/2145] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:18:50.359849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:50.361347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:50.435287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411f666-c93c-4733-b781-ade9fa420ed3 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:50.440484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d62224-299e-4755-af7d-c41be0b49e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:50.441134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d62224-299e-4755-af7d-c41be0b49e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:50.441419 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d62224-299e-4755-af7d-c41be0b49e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:50.441698 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d62224-299e-4755-af7d-c41be0b49e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:50.442332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d62224-299e-4755-af7d-c41be0b49e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:50.444282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d62224-299e-4755-af7d-c41be0b49e8d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:50.536556 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1077/2146] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:50 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:50.537669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83886a32-ca11-49f0-b99f-fef61ddcd944 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:50.548059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83886a32-ca11-49f0-b99f-fef61ddcd944 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:50.617727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7b5b7-6f0c-4a0d-a481-ef04846cfc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:50.618339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7b5b7-6f0c-4a0d-a481-ef04846cfc3a admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:50.618618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7b5b7-6f0c-4a0d-a481-ef04846cfc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:50.618890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7b5b7-6f0c-4a0d-a481-ef04846cfc3a admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:50.619520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7b5b7-6f0c-4a0d-a481-ef04846cfc3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:50.622365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7b5b7-6f0c-4a0d-a481-ef04846cfc3a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:50.750600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1070/2147] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:50 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:18:50.755382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d62224-299e-4755-af7d-c41be0b49e8d admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:50.766246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d62224-299e-4755-af7d-c41be0b49e8d admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:50.771363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf9734-231c-4cba-9dd1-0d14316cc2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:50.772020 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf9734-231c-4cba-9dd1-0d14316cc2dc admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:50.772305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf9734-231c-4cba-9dd1-0d14316cc2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:50.772608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf9734-231c-4cba-9dd1-0d14316cc2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:50.773407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf9734-231c-4cba-9dd1-0d14316cc2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:50.781493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf9734-231c-4cba-9dd1-0d14316cc2dc admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:50.790196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1078/2148] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:50.792773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7b5b7-6f0c-4a0d-a481-ef04846cfc3a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:50.803334 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7b5b7-6f0c-4a0d-a481-ef04846cfc3a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:50.808965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2e560-d61c-4dc1-80ef-e3644961f689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:50.808965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2e560-d61c-4dc1-80ef-e3644961f689 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:50.809187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2e560-d61c-4dc1-80ef-e3644961f689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:50.809411 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2e560-d61c-4dc1-80ef-e3644961f689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:50.810054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2e560-d61c-4dc1-80ef-e3644961f689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:50.812081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2e560-d61c-4dc1-80ef-e3644961f689 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:50.819638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf9734-231c-4cba-9dd1-0d14316cc2dc admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:50.825033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-1cdf9734-231c-4cba-9dd1-0d14316cc2dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:50.826954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1071/2149] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:18:50.827829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2e560-d61c-4dc1-80ef-e3644961f689 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:50.837867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2e560-d61c-4dc1-80ef-e3644961f689 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:50.841681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf9734-231c-4cba-9dd1-0d14316cc2dc admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:50.843834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc9eb3-8c72-47b7-812a-06c9ced43562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:50.844269 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc9eb3-8c72-47b7-812a-06c9ced43562 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:50.844555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc9eb3-8c72-47b7-812a-06c9ced43562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:50.844830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc9eb3-8c72-47b7-812a-06c9ced43562 admin admin] PATH_INFO: `/v3/projects/5cd097a59291481ab5cd35689765e9bc/users/dff7bd7a1a3f48b696c0e36a6d15757a/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:50.854284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:50.855460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:50.855819 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:50.856116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] PATH_INFO: `/v3/projects/0b91db6b85c543b38be6c109874d944b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:50.857639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b91db6b85c543b38be6c109874d944b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:50.859566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:50.860910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc9eb3-8c72-47b7-812a-06c9ced43562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd097a59291481ab5cd35689765e9bc, user_id=dff7bd7a1a3f48b696c0e36a6d15757a, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:50.863574 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc9eb3-8c72-47b7-812a-06c9ced43562 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:50.932469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1072/2150] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:50 2022] PUT /identity/v3/projects/5cd097a59291481ab5cd35689765e9bc/users/dff7bd7a1a3f48b696c0e36a6d15757a/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:50.937742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5e044-7750-48e8-aaa5-2c204a894771 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:50.938035 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5e044-7750-48e8-aaa5-2c204a894771 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:50.938342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5e044-7750-48e8-aaa5-2c204a894771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:50.957720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-91a5e044-7750-48e8-aaa5-2c204a894771 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:50.959002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-91a5e044-7750-48e8-aaa5-2c204a894771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:50.966629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b91db6b85c543b38be6c109874d944b', 'request_id': 'req-508e70ea-7cf1-442a-8fa2-64af9ef2a401'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:50.972448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b91db6b85c543b38be6c109874d944b', 'request_id': 'req-508e70ea-7cf1-442a-8fa2-64af9ef2a401'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:51.022264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1073/2151] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:51.032616 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5e044-7750-48e8-aaa5-2c204a894771 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:51.033994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-91a5e044-7750-48e8-aaa5-2c204a894771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:51.067939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b91db6b85c543b38be6c109874d944b', 'request_id': 'req-508e70ea-7cf1-442a-8fa2-64af9ef2a401'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:51.085562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] The token cache is being invalidate because project 0b91db6b85c543b38be6c109874d944b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:51.085859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:51.086520 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-508e70ea-7cf1-442a-8fa2-64af9ef2a401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:51.104513 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1079/2152] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:50 2022] DELETE /identity/v3/projects/0b91db6b85c543b38be6c109874d944b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:51.105253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5e044-7750-48e8-aaa5-2c204a894771 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:51.110076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d4365-1bf9-41d6-a30d-c88570076267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:51.112239 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d4365-1bf9-41d6-a30d-c88570076267 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:51.113636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d4365-1bf9-41d6-a30d-c88570076267 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:51.115251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d4365-1bf9-41d6-a30d-c88570076267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:51.119119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-405d4365-1bf9-41d6-a30d-c88570076267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:51.148389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:51.149044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d4365-1bf9-41d6-a30d-c88570076267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:51.150827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d4365-1bf9-41d6-a30d-c88570076267 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:51.174386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1074/2153] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:18:51.311156 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:53.011582 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef342ff-35a5-4c9a-ba8c-e38c41037dda None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:53.012064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef342ff-35a5-4c9a-ba8c-e38c41037dda None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:53.012411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef342ff-35a5-4c9a-ba8c-e38c41037dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:53.042238 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-6ef342ff-35a5-4c9a-ba8c-e38c41037dda None None] MFA Rules not processed for user `dff7bd7a1a3f48b696c0e36a6d15757a`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:53.044102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6ef342ff-35a5-4c9a-ba8c-e38c41037dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:53.144680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1080/2154] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:53.156669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d4365-1bf9-41d6-a30d-c88570076267 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:53.158200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-405d4365-1bf9-41d6-a30d-c88570076267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:53.220827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d4365-1bf9-41d6-a30d-c88570076267 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:53.225010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6462123-11f4-4c01-8fbe-26707480b6a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:53.225727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6462123-11f4-4c01-8fbe-26707480b6a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:53.226107 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6462123-11f4-4c01-8fbe-26707480b6a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:53.226480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6462123-11f4-4c01-8fbe-26707480b6a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:53.236583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6462123-11f4-4c01-8fbe-26707480b6a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:53.238189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6462123-11f4-4c01-8fbe-26707480b6a5 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:53.250009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1075/2155] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:55.447014 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cd192-ab59-49e6-a970-19275daf9bbe None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:55.447487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cd192-ab59-49e6-a970-19275daf9bbe None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:55.447575 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cd192-ab59-49e6-a970-19275daf9bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:55.466881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-794cd192-ab59-49e6-a970-19275daf9bbe None None] MFA Rules not processed for user `dff7bd7a1a3f48b696c0e36a6d15757a`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:55.468152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-794cd192-ab59-49e6-a970-19275daf9bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:55.475454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1081/2156] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:18:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:03.246517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=93,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:04.431205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d526e-1dc7-4386-a203-98398f4b7965 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:04.431825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d526e-1dc7-4386-a203-98398f4b7965 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:04.431825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d526e-1dc7-4386-a203-98398f4b7965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:04.454749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-011d526e-1dc7-4386-a203-98398f4b7965 None None] MFA Rules not processed for user `c94c65ae14e94796a3b3bd0bb29a91b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:04.456036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-011d526e-1dc7-4386-a203-98398f4b7965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:04.518636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1076/2157] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:19:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:05.095696 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45691178-2861-4578-9785-98ff37425c5d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:05.096185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45691178-2861-4578-9785-98ff37425c5d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:05.096389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45691178-2861-4578-9785-98ff37425c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:05.122882 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-45691178-2861-4578-9785-98ff37425c5d None None] MFA Rules not processed for user `388174c87a2245ca89a634f3b177e166`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:05.124109 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-45691178-2861-4578-9785-98ff37425c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:05.187385 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:05.195226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1082/2158] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:19:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:14.510631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=25 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:18.494667 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a0ce0-b44f-4344-b621-d86253fca897 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:18.495253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a0ce0-b44f-4344-b621-d86253fca897 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:18.495253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a0ce0-b44f-4344-b621-d86253fca897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:18.514896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-8f8a0ce0-b44f-4344-b621-d86253fca897 None None] MFA Rules not processed for user `dff7bd7a1a3f48b696c0e36a6d15757a`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:18.516084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8f8a0ce0-b44f-4344-b621-d86253fca897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:18.522907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1077/2159] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:19:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:28.509456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:48.233448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf5f3-d5fa-478b-9b77-ffbac6af1509 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:48.233968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf5f3-d5fa-478b-9b77-ffbac6af1509 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:48.234070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf5f3-d5fa-478b-9b77-ffbac6af1509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:48.255204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-882bf5f3-d5fa-478b-9b77-ffbac6af1509 None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:48.257673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-882bf5f3-d5fa-478b-9b77-ffbac6af1509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:48.313686 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1083/2160] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:19:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:48.355459 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a0ce0-b44f-4344-b621-d86253fca897 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:48.356848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8f8a0ce0-b44f-4344-b621-d86253fca897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:48.425266 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a0ce0-b44f-4344-b621-d86253fca897 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:48.430270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e200ab3-71db-47c6-bae9-2bb4312669bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:48.430873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e200ab3-71db-47c6-bae9-2bb4312669bb service swift] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:48.431154 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e200ab3-71db-47c6-bae9-2bb4312669bb service swift] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:48.431430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e200ab3-71db-47c6-bae9-2bb4312669bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:48.439466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e200ab3-71db-47c6-bae9-2bb4312669bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:48.441064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e200ab3-71db-47c6-bae9-2bb4312669bb service swift] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:48.449547 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1078/2161] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:19:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:19:58.307017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=25 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:58.446351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=32 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:05.631399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d0fdb-7fb2-4a66-bbdc-39b0cb2df2ba None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.631885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d0fdb-7fb2-4a66-bbdc-39b0cb2df2ba None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.632027 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d0fdb-7fb2-4a66-bbdc-39b0cb2df2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.652367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-d42d0fdb-7fb2-4a66-bbdc-39b0cb2df2ba None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:05.653687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d42d0fdb-7fb2-4a66-bbdc-39b0cb2df2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:05.660992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1084/2162] 10.222.0.30 () {58 vars in 967 bytes} [Wed Aug 24 10:20:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:05.673313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e200ab3-71db-47c6-bae9-2bb4312669bb service swift] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:05.684023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e200ab3-71db-47c6-bae9-2bb4312669bb service swift] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:05.689019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbfb90-98c6-4d62-b494-8f099daa0496 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:05.689617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbfb90-98c6-4d62-b494-8f099daa0496 service swift] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.689890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbfb90-98c6-4d62-b494-8f099daa0496 service swift] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.690169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbfb90-98c6-4d62-b494-8f099daa0496 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.699256 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbfb90-98c6-4d62-b494-8f099daa0496 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:05.700823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbfb90-98c6-4d62-b494-8f099daa0496 service swift] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:05.709904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1079/2163] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:20:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:20:06.866581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74949eb-5489-488c-860e-c88c79a073d4 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:06.867057 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74949eb-5489-488c-860e-c88c79a073d4 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:06.867155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74949eb-5489-488c-860e-c88c79a073d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:06.886577 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b74949eb-5489-488c-860e-c88c79a073d4 None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:06.888054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b74949eb-5489-488c-860e-c88c79a073d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:06.897240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1085/2164] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:20:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:06.914242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbfb90-98c6-4d62-b494-8f099daa0496 service swift] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:06.925310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbfb90-98c6-4d62-b494-8f099daa0496 service swift] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:06.930912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8717cf1-2c65-4ea4-88ff-d2ba717fc334 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:06.931515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717cf1-2c65-4ea4-88ff-d2ba717fc334 service swift] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:06.931797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717cf1-2c65-4ea4-88ff-d2ba717fc334 service swift] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:06.932071 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717cf1-2c65-4ea4-88ff-d2ba717fc334 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:06.942491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8717cf1-2c65-4ea4-88ff-d2ba717fc334 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:06.945441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8717cf1-2c65-4ea4-88ff-d2ba717fc334 service swift] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:06.954366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1080/2165] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:20:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:20:08.977869 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898f19-fd8f-4c11-b9e3-5455b897bacd None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:08.978866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898f19-fd8f-4c11-b9e3-5455b897bacd None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:08.979329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898f19-fd8f-4c11-b9e3-5455b897bacd None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:08.980877 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1086/2166] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:20:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:20:08.998763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fef97c-3f4e-4601-b8aa-1afaf6e75458 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:08.999654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fef97c-3f4e-4601-b8aa-1afaf6e75458 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:09.011659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fef97c-3f4e-4601-b8aa-1afaf6e75458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:09.033222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-40fef97c-3f4e-4601-b8aa-1afaf6e75458 None None] MFA Rules not processed for user `b1aa47a883604336a8ab90d324470477`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:09.035181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-40fef97c-3f4e-4601-b8aa-1afaf6e75458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:09.137832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1081/2167] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:20:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:09.159406 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df898f19-fd8f-4c11-b9e3-5455b897bacd None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:09.178492 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-df898f19-fd8f-4c11-b9e3-5455b897bacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:09.218123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df898f19-fd8f-4c11-b9e3-5455b897bacd None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:09.226238 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8f8c3-0d82-4f55-adda-0936ef49ba5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:09.229271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8f8c3-0d82-4f55-adda-0936ef49ba5d service placement] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:09.229899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8f8c3-0d82-4f55-adda-0936ef49ba5d service placement] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:09.230555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8f8c3-0d82-4f55-adda-0936ef49ba5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:09.255563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8f8c3-0d82-4f55-adda-0936ef49ba5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:09.259035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8f8c3-0d82-4f55-adda-0936ef49ba5d service placement] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:09.308640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fef97c-3f4e-4601-b8aa-1afaf6e75458 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:09.310312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-40fef97c-3f4e-4601-b8aa-1afaf6e75458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:09.311510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1087/2168] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:09.331034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fef97c-3f4e-4601-b8aa-1afaf6e75458 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:09.343298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeaf472-6310-47ba-be65-ad6af0e1b3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:09.345211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf472-6310-47ba-be65-ad6af0e1b3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:09.345500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf472-6310-47ba-be65-ad6af0e1b3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:09.345785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf472-6310-47ba-be65-ad6af0e1b3ff admin admin] PATH_INFO: `/v3/users/dff7bd7a1a3f48b696c0e36a6d15757a` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:09.346984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeaf472-6310-47ba-be65-ad6af0e1b3ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=dff7bd7a1a3f48b696c0e36a6d15757a)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:09.349624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeaf472-6310-47ba-be65-ad6af0e1b3ff admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:10.122981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-edeaf472-6310-47ba-be65-ad6af0e1b3ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dff7bd7a1a3f48b696c0e36a6d15757a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:10.123709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-edeaf472-6310-47ba-be65-ad6af0e1b3ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dff7bd7a1a3f48b696c0e36a6d15757a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:10.131548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-edeaf472-6310-47ba-be65-ad6af0e1b3ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dff7bd7a1a3f48b696c0e36a6d15757a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:10.132193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-edeaf472-6310-47ba-be65-ad6af0e1b3ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dff7bd7a1a3f48b696c0e36a6d15757a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:10.279660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-edeaf472-6310-47ba-be65-ad6af0e1b3ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dff7bd7a1a3f48b696c0e36a6d15757a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:10.302490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1082/2169] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:20:09 2022] DELETE /identity/v3/users/dff7bd7a1a3f48b696c0e36a6d15757a => generated 0 bytes in 994 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:10.659190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8f8c3-0d82-4f55-adda-0936ef49ba5d service placement] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:10.678518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8f8c3-0d82-4f55-adda-0936ef49ba5d service placement] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:10.799026 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:10.799679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:10.799957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:10.800229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] PATH_INFO: `/v3/projects/5cd097a59291481ab5cd35689765e9bc` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:10.802977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cd097a59291481ab5cd35689765e9bc)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:10.805087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:10.944940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd097a59291481ab5cd35689765e9bc', 'request_id': 'req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:10.953006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd097a59291481ab5cd35689765e9bc', 'request_id': 'req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:11.039893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd097a59291481ab5cd35689765e9bc', 'request_id': 'req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:11.055894 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] The token cache is being invalidate because project 5cd097a59291481ab5cd35689765e9bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:20:11.056192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:11.056953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c4d15c5a-5ed4-4e45-93a4-955fd72b5bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:11.058484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1088/2170] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:20:10 2022] DELETE /identity/v3/projects/5cd097a59291481ab5cd35689765e9bc => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:11.063565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39947604-28ed-4913-bc47-3b3865a872bb None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:11.064174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39947604-28ed-4913-bc47-3b3865a872bb None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:11.064772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39947604-28ed-4913-bc47-3b3865a872bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:11.088159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-39947604-28ed-4913-bc47-3b3865a872bb None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:11.089797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-39947604-28ed-4913-bc47-3b3865a872bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:11.133760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:11.158675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1083/2171] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:20:11.167829 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978834f4-6156-48e8-a11b-7b102533453a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:11.168115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978834f4-6156-48e8-a11b-7b102533453a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:11.168392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978834f4-6156-48e8-a11b-7b102533453a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:11.190747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-978834f4-6156-48e8-a11b-7b102533453a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:11.192119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-978834f4-6156-48e8-a11b-7b102533453a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:11.202482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1089/2172] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:11.204672 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39947604-28ed-4913-bc47-3b3865a872bb None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:11.217429 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39947604-28ed-4913-bc47-3b3865a872bb None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:11.229051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5598cf-4329-49b8-abe3-c752354bcc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:11.229051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5598cf-4329-49b8-abe3-c752354bcc89 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:11.229051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5598cf-4329-49b8-abe3-c752354bcc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:11.229051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5598cf-4329-49b8-abe3-c752354bcc89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:11.229545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5598cf-4329-49b8-abe3-c752354bcc89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:20:11.229545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5598cf-4329-49b8-abe3-c752354bcc89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:11.229545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5598cf-4329-49b8-abe3-c752354bcc89 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:11.243916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1084/2173] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:20:11.247595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978834f4-6156-48e8-a11b-7b102533453a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:11.258583 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978834f4-6156-48e8-a11b-7b102533453a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:11.271497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97cab3-8bc3-4e97-80b5-223db0f5ad8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:11.272320 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97cab3-8bc3-4e97-80b5-223db0f5ad8e admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:11.272627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97cab3-8bc3-4e97-80b5-223db0f5ad8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:11.272942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97cab3-8bc3-4e97-80b5-223db0f5ad8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:11.273591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97cab3-8bc3-4e97-80b5-223db0f5ad8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:11.275537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97cab3-8bc3-4e97-80b5-223db0f5ad8e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:11.357967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1090/2174] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:20:11 2022] POST /identity/v3/projects => generated 389 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:20:11.363078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5598cf-4329-49b8-abe3-c752354bcc89 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:11.375925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5598cf-4329-49b8-abe3-c752354bcc89 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:11.462817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f690f-1039-48c9-a8e1-8667ffc6e72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:11.463492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f690f-1039-48c9-a8e1-8667ffc6e72e admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:11.463768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f690f-1039-48c9-a8e1-8667ffc6e72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:11.464206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f690f-1039-48c9-a8e1-8667ffc6e72e admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:11.464946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f690f-1039-48c9-a8e1-8667ffc6e72e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:11.467147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f690f-1039-48c9-a8e1-8667ffc6e72e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:11.583223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1085/2175] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:20:11 2022] POST /identity/v3/users => generated 295 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:20:11.587479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97cab3-8bc3-4e97-80b5-223db0f5ad8e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:11.601331 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97cab3-8bc3-4e97-80b5-223db0f5ad8e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:11.606805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211cba56-76e9-405d-88e3-3748888aa39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:11.607615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cba56-76e9-405d-88e3-3748888aa39e admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:11.608054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cba56-76e9-405d-88e3-3748888aa39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:11.608524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cba56-76e9-405d-88e3-3748888aa39e admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:11.609659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211cba56-76e9-405d-88e3-3748888aa39e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:11.611446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211cba56-76e9-405d-88e3-3748888aa39e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:11.628477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1091/2176] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:20:11.632952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f690f-1039-48c9-a8e1-8667ffc6e72e admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:11.647271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f690f-1039-48c9-a8e1-8667ffc6e72e admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:11.652301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09028a38-b7f7-46a6-bdc1-c01df764368f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:11.652937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09028a38-b7f7-46a6-bdc1-c01df764368f admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:11.653224 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09028a38-b7f7-46a6-bdc1-c01df764368f admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:11.653801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09028a38-b7f7-46a6-bdc1-c01df764368f admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:11.654456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09028a38-b7f7-46a6-bdc1-c01df764368f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:11.656580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09028a38-b7f7-46a6-bdc1-c01df764368f admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:11.671461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1086/2177] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:20:11.675355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211cba56-76e9-405d-88e3-3748888aa39e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:11.694378 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211cba56-76e9-405d-88e3-3748888aa39e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:11.700329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c82aff-eec5-4acf-a40d-d4cfd4732edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:11.701429 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c82aff-eec5-4acf-a40d-d4cfd4732edd admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:11.701819 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c82aff-eec5-4acf-a40d-d4cfd4732edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:11.702274 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c82aff-eec5-4acf-a40d-d4cfd4732edd admin admin] PATH_INFO: `/v3/projects/fd7133fef3174d538c494a10bbb8d52f/users/73cf66f5088143ec84da4133423f1d87/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:11.721038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c82aff-eec5-4acf-a40d-d4cfd4732edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7133fef3174d538c494a10bbb8d52f, user_id=73cf66f5088143ec84da4133423f1d87, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:11.725342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c82aff-eec5-4acf-a40d-d4cfd4732edd admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:11.831485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:11.840895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1092/2178] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:20:11 2022] PUT /identity/v3/projects/fd7133fef3174d538c494a10bbb8d52f/users/73cf66f5088143ec84da4133423f1d87/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:11.856578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbe9b0-4b8b-458e-a897-467501a909d2 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:11.857023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbe9b0-4b8b-458e-a897-467501a909d2 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:11.857786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbe9b0-4b8b-458e-a897-467501a909d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:11.924184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-56dbe9b0-4b8b-458e-a897-467501a909d2 None None] MFA Rules not processed for user `73cf66f5088143ec84da4133423f1d87`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:11.925659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-56dbe9b0-4b8b-458e-a897-467501a909d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:12.033848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1087/2179] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:12.045884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c82aff-eec5-4acf-a40d-d4cfd4732edd admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:12.048446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c8c82aff-eec5-4acf-a40d-d4cfd4732edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:12.136355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c82aff-eec5-4acf-a40d-d4cfd4732edd admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:12.148036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e115eee-7718-4774-99e4-0dd715064ef1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:12.148932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e115eee-7718-4774-99e4-0dd715064ef1 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:12.149376 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e115eee-7718-4774-99e4-0dd715064ef1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:12.149823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e115eee-7718-4774-99e4-0dd715064ef1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:12.159731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e115eee-7718-4774-99e4-0dd715064ef1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:12.161275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e115eee-7718-4774-99e4-0dd715064ef1 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:12.170834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1093/2180] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:17.989439 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbe9b0-4b8b-458e-a897-467501a909d2 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:17.991048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-56dbe9b0-4b8b-458e-a897-467501a909d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:18.101178 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:18.170200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbe9b0-4b8b-458e-a897-467501a909d2 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:18.207215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05b1f8-1920-470c-8dc0-3957de165c9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:18.208021 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05b1f8-1920-470c-8dc0-3957de165c9d service placement] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:18.208307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05b1f8-1920-470c-8dc0-3957de165c9d service placement] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:18.208585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05b1f8-1920-470c-8dc0-3957de165c9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:18.210931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9c05b1f8-1920-470c-8dc0-3957de165c9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:18.231253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05b1f8-1920-470c-8dc0-3957de165c9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:18.233099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05b1f8-1920-470c-8dc0-3957de165c9d service placement] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:18.245391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1088/2181] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:22.222836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=26 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:28.308379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=15 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:29.836728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d1864-5076-4117-93e8-65bdb090ea55 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:29.837310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d1864-5076-4117-93e8-65bdb090ea55 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:29.837420 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d1864-5076-4117-93e8-65bdb090ea55 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:29.838739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1094/2182] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:20:29.844884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce704d-e8fc-432f-ab1d-2fb09c031257 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:29.845385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce704d-e8fc-432f-ab1d-2fb09c031257 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:29.845606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce704d-e8fc-432f-ab1d-2fb09c031257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:29.867302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e0ce704d-e8fc-432f-ab1d-2fb09c031257 None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:29.868509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e0ce704d-e8fc-432f-ab1d-2fb09c031257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:29.941166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1089/2183] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:29.956709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d1864-5076-4117-93e8-65bdb090ea55 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:29.956709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e23d1864-5076-4117-93e8-65bdb090ea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:30.025608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d1864-5076-4117-93e8-65bdb090ea55 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:30.029352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e2096-7b3b-4156-80a5-2b754620e5e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0b124ca267b4a4391557ad1396524cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:30.030036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e2096-7b3b-4156-80a5-2b754620e5e7 service glance] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:30.030315 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e2096-7b3b-4156-80a5-2b754620e5e7 service glance] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:30.030633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e2096-7b3b-4156-80a5-2b754620e5e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:30.041763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e2096-7b3b-4156-80a5-2b754620e5e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:30.043236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e2096-7b3b-4156-80a5-2b754620e5e7 service glance] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:30.055087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1095/2184] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:30.281976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873de5a-f6f4-4373-8643-d2c9f5ce8f01 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:30.282166 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873de5a-f6f4-4373-8643-d2c9f5ce8f01 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:30.282448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873de5a-f6f4-4373-8643-d2c9f5ce8f01 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:30.283926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1090/2185] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:20:30.290623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eda1a0-0e7d-4a03-80fd-1970c472cc4f None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:30.290914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eda1a0-0e7d-4a03-80fd-1970c472cc4f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:30.291197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eda1a0-0e7d-4a03-80fd-1970c472cc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:30.312428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-d8eda1a0-0e7d-4a03-80fd-1970c472cc4f None None] MFA Rules not processed for user `ae9b6525009b41288dd918437acee92a`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:30.317128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d8eda1a0-0e7d-4a03-80fd-1970c472cc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:30.369307 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1096/2186] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:20:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:30.381843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873de5a-f6f4-4373-8643-d2c9f5ce8f01 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:30.383988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8873de5a-f6f4-4373-8643-d2c9f5ce8f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:30.398093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873de5a-f6f4-4373-8643-d2c9f5ce8f01 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:30.401625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022c824-aff3-418b-ae65-081cebfc260d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:30.402232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022c824-aff3-418b-ae65-081cebfc260d service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:30.402515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022c824-aff3-418b-ae65-081cebfc260d service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:30.402790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022c824-aff3-418b-ae65-081cebfc260d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:30.411455 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022c824-aff3-418b-ae65-081cebfc260d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:30.414672 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022c824-aff3-418b-ae65-081cebfc260d service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:30.423217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1091/2187] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:30.512069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba1866-36b6-412d-95e9-0f7567ee7df2 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:30.512409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba1866-36b6-412d-95e9-0f7567ee7df2 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:30.512628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba1866-36b6-412d-95e9-0f7567ee7df2 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:30.514045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1097/2188] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:20:30.520065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445aa89-e31c-4327-9288-8a91e43d7c6e None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:30.520358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445aa89-e31c-4327-9288-8a91e43d7c6e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:30.520702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445aa89-e31c-4327-9288-8a91e43d7c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:30.542311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1445aa89-e31c-4327-9288-8a91e43d7c6e None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:30.543515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1445aa89-e31c-4327-9288-8a91e43d7c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:30.554136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1092/2189] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:20:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:30.567768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba1866-36b6-412d-95e9-0f7567ee7df2 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:30.570388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-35ba1866-36b6-412d-95e9-0f7567ee7df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:30.584221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba1866-36b6-412d-95e9-0f7567ee7df2 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:30.588155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db4d97-5e83-4658-a091-01d8b553fda6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:30.588882 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db4d97-5e83-4658-a091-01d8b553fda6 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:30.589164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db4d97-5e83-4658-a091-01d8b553fda6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:30.589563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db4d97-5e83-4658-a091-01d8b553fda6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:30.601219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db4d97-5e83-4658-a091-01d8b553fda6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:30.602775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db4d97-5e83-4658-a091-01d8b553fda6 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:30.617093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1098/2190] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:33.670797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445aa89-e31c-4327-9288-8a91e43d7c6e None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:33.733599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445aa89-e31c-4327-9288-8a91e43d7c6e None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:33.738027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:33.738909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:33.739190 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:33.739458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] PATH_INFO: `/v3/users/73cf66f5088143ec84da4133423f1d87` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:33.740751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=73cf66f5088143ec84da4133423f1d87)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:33.742704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:33.926444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73cf66f5088143ec84da4133423f1d87'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:33.927103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73cf66f5088143ec84da4133423f1d87'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:33.934539 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73cf66f5088143ec84da4133423f1d87'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:33.935195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73cf66f5088143ec84da4133423f1d87'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:34.005359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73cf66f5088143ec84da4133423f1d87'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:34.015053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:34.025777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1093/2191] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:20:33 2022] DELETE /identity/v3/users/73cf66f5088143ec84da4133423f1d87 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:34.030545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dce6d-4637-48a4-874c-3250f842219e None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:34.031016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dce6d-4637-48a4-874c-3250f842219e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:34.031135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dce6d-4637-48a4-874c-3250f842219e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:34.051389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-034dce6d-4637-48a4-874c-3250f842219e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:34.052650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-034dce6d-4637-48a4-874c-3250f842219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:34.121053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1099/2192] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:34.134938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:34.136707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:34.202497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08243938-073e-40e5-a52d-5883e3d3b2dd admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:34.207360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfff96-0a91-4c71-b745-2149cba6ef17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:34.208093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfff96-0a91-4c71-b745-2149cba6ef17 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:34.208479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfff96-0a91-4c71-b745-2149cba6ef17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:34.209645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfff96-0a91-4c71-b745-2149cba6ef17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:34.221457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfff96-0a91-4c71-b745-2149cba6ef17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:34.223769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfff96-0a91-4c71-b745-2149cba6ef17 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:34.240190 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1094/2193] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:34.772749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034dce6d-4637-48a4-874c-3250f842219e None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:34.774350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-034dce6d-4637-48a4-874c-3250f842219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:34.793796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034dce6d-4637-48a4-874c-3250f842219e None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:34.799089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:34.799816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:34.800102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:34.800384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] PATH_INFO: `/v3/projects/fd7133fef3174d538c494a10bbb8d52f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:34.801532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7133fef3174d538c494a10bbb8d52f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:34.804907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:34.982657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:34.997072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7133fef3174d538c494a10bbb8d52f', 'request_id': 'req-bc429649-85bd-4c2c-9e98-41d31911a8be'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:35.004576 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7133fef3174d538c494a10bbb8d52f', 'request_id': 'req-bc429649-85bd-4c2c-9e98-41d31911a8be'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:35.105870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7133fef3174d538c494a10bbb8d52f', 'request_id': 'req-bc429649-85bd-4c2c-9e98-41d31911a8be'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:35.120117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] The token cache is being invalidate because project fd7133fef3174d538c494a10bbb8d52f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:20:35.124517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:35.124517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bc429649-85bd-4c2c-9e98-41d31911a8be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:35.124517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1100/2194] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:20:34 2022] DELETE /identity/v3/projects/fd7133fef3174d538c494a10bbb8d52f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:35.129258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d7df6-66f6-4359-a7ab-f96750c6ed16 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:35.129808 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d7df6-66f6-4359-a7ab-f96750c6ed16 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:35.129808 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d7df6-66f6-4359-a7ab-f96750c6ed16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:35.156273 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-d78d7df6-66f6-4359-a7ab-f96750c6ed16 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:35.158935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d78d7df6-66f6-4359-a7ab-f96750c6ed16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:35.226213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1095/2195] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:35.234534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3da5cd-f0ef-41ec-8dba-88fc887fd226 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:35.234828 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3da5cd-f0ef-41ec-8dba-88fc887fd226 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:35.235107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3da5cd-f0ef-41ec-8dba-88fc887fd226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:35.263675 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0f3da5cd-f0ef-41ec-8dba-88fc887fd226 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:35.264836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0f3da5cd-f0ef-41ec-8dba-88fc887fd226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:35.273818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1101/2196] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:35.278131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d7df6-66f6-4359-a7ab-f96750c6ed16 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:35.290932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d7df6-66f6-4359-a7ab-f96750c6ed16 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:35.303015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2eb3f-f21c-4b88-a411-bea04cc50c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:35.303645 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2eb3f-f21c-4b88-a411-bea04cc50c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:35.304002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2eb3f-f21c-4b88-a411-bea04cc50c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:35.304280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2eb3f-f21c-4b88-a411-bea04cc50c37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:35.305383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2eb3f-f21c-4b88-a411-bea04cc50c37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:20:35.305700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2eb3f-f21c-4b88-a411-bea04cc50c37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:35.307408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2eb3f-f21c-4b88-a411-bea04cc50c37 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:35.324914 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1096/2197] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:20:35.326166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3da5cd-f0ef-41ec-8dba-88fc887fd226 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:35.336717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3da5cd-f0ef-41ec-8dba-88fc887fd226 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:35.341742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a88364-9ac8-43a0-9e65-4324390b9f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:35.342433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a88364-9ac8-43a0-9e65-4324390b9f67 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:35.342718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a88364-9ac8-43a0-9e65-4324390b9f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:35.342992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a88364-9ac8-43a0-9e65-4324390b9f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:35.343680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a88364-9ac8-43a0-9e65-4324390b9f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:35.345923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a88364-9ac8-43a0-9e65-4324390b9f67 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:35.478568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1102/2198] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:20:35 2022] POST /identity/v3/projects => generated 369 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:20:35.479887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2eb3f-f21c-4b88-a411-bea04cc50c37 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:35.490384 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2eb3f-f21c-4b88-a411-bea04cc50c37 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:35.562665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6cc95-8ee8-412c-9ee5-a2499a381b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:35.563396 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6cc95-8ee8-412c-9ee5-a2499a381b32 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:35.563820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6cc95-8ee8-412c-9ee5-a2499a381b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:35.564020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6cc95-8ee8-412c-9ee5-a2499a381b32 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:35.565074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6cc95-8ee8-412c-9ee5-a2499a381b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:35.567061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6cc95-8ee8-412c-9ee5-a2499a381b32 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:35.706777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1097/2199] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:20:35 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:20:35.708302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a88364-9ac8-43a0-9e65-4324390b9f67 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:35.722797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a88364-9ac8-43a0-9e65-4324390b9f67 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:35.727630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617792d-fd72-4e8c-9a2f-54db78d4305f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:35.728171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617792d-fd72-4e8c-9a2f-54db78d4305f admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:35.728451 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617792d-fd72-4e8c-9a2f-54db78d4305f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:35.728725 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617792d-fd72-4e8c-9a2f-54db78d4305f admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:35.729647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617792d-fd72-4e8c-9a2f-54db78d4305f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:35.731990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617792d-fd72-4e8c-9a2f-54db78d4305f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:35.746086 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1103/2200] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:20:35.750219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6cc95-8ee8-412c-9ee5-a2499a381b32 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:35.773233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6cc95-8ee8-412c-9ee5-a2499a381b32 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:35.784284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6307dc1-1719-4a37-9752-ab6904e54dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:35.784946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6307dc1-1719-4a37-9752-ab6904e54dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:35.785235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6307dc1-1719-4a37-9752-ab6904e54dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:35.785525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6307dc1-1719-4a37-9752-ab6904e54dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:35.786420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6307dc1-1719-4a37-9752-ab6904e54dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:35.788219 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6307dc1-1719-4a37-9752-ab6904e54dd2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:35.794931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:35.801713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1098/2201] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:20:35.806755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617792d-fd72-4e8c-9a2f-54db78d4305f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:35.831542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617792d-fd72-4e8c-9a2f-54db78d4305f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:35.842776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6afdd7-b41d-4523-8f06-f51645610670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:35.843458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6afdd7-b41d-4523-8f06-f51645610670 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:35.843740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6afdd7-b41d-4523-8f06-f51645610670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:35.844026 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6afdd7-b41d-4523-8f06-f51645610670 admin admin] PATH_INFO: `/v3/projects/f2c601f7e6b64f83919d960b601e9331/users/c28281bb90464004a751699a49c61ebf/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:35.873094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6afdd7-b41d-4523-8f06-f51645610670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c601f7e6b64f83919d960b601e9331, user_id=c28281bb90464004a751699a49c61ebf, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:35.876042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6afdd7-b41d-4523-8f06-f51645610670 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:35.952189 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bb352-b318-4e0b-bb73-61adec668897 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:35.952493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bb352-b318-4e0b-bb73-61adec668897 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:35.952881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bb352-b318-4e0b-bb73-61adec668897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:35.956452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1104/2202] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:20:35 2022] PUT /identity/v3/projects/f2c601f7e6b64f83919d960b601e9331/users/c28281bb90464004a751699a49c61ebf/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:35.978108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-152bb352-b318-4e0b-bb73-61adec668897 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:35.979484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-152bb352-b318-4e0b-bb73-61adec668897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:36.054940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1099/2203] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:36.065562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6afdd7-b41d-4523-8f06-f51645610670 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:36.068099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cb6afdd7-b41d-4523-8f06-f51645610670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:36.145034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6afdd7-b41d-4523-8f06-f51645610670 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:36.150968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d452e6-3164-48cd-a9ee-902bd0ce1a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:36.151929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d452e6-3164-48cd-a9ee-902bd0ce1a5e service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:36.152674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d452e6-3164-48cd-a9ee-902bd0ce1a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:36.153352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d452e6-3164-48cd-a9ee-902bd0ce1a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:36.166504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d452e6-3164-48cd-a9ee-902bd0ce1a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:36.168437 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d452e6-3164-48cd-a9ee-902bd0ce1a5e service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:36.182568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1105/2204] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:37.339609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fc99c-13ec-4f21-9ccc-c6f4a98d1db8 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:37.340114 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fc99c-13ec-4f21-9ccc-c6f4a98d1db8 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:37.340228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fc99c-13ec-4f21-9ccc-c6f4a98d1db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:37.362558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-b67fc99c-13ec-4f21-9ccc-c6f4a98d1db8 None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:37.362558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b67fc99c-13ec-4f21-9ccc-c6f4a98d1db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:37.441214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1100/2205] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:20:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:37.456000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d452e6-3164-48cd-a9ee-902bd0ce1a5e service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:37.458654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b1d452e6-3164-48cd-a9ee-902bd0ce1a5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:37.564757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:37.569508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d452e6-3164-48cd-a9ee-902bd0ce1a5e service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:37.574483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16378697-67af-439f-9acc-3b7eec24e54b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:37.575087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378697-67af-439f-9acc-3b7eec24e54b service swift] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:37.575361 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378697-67af-439f-9acc-3b7eec24e54b service swift] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:37.575630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378697-67af-439f-9acc-3b7eec24e54b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:37.584461 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16378697-67af-439f-9acc-3b7eec24e54b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:37.586196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16378697-67af-439f-9acc-3b7eec24e54b service swift] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:37.594119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1106/2206] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:20:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:20:42.156058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8b776-be5f-44b5-84fd-3dca5f2ffa95 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:42.156724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8b776-be5f-44b5-84fd-3dca5f2ffa95 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:42.157339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8b776-be5f-44b5-84fd-3dca5f2ffa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:42.226429 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a8a8b776-be5f-44b5-84fd-3dca5f2ffa95 None None] MFA Rules not processed for user `c94c65ae14e94796a3b3bd0bb29a91b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:42.227964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a8a8b776-be5f-44b5-84fd-3dca5f2ffa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:42.340196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1101/2207] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:46.315296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af98333-66f3-4123-af6d-3b8608bb7b7d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:46.315803 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af98333-66f3-4123-af6d-3b8608bb7b7d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:46.315903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af98333-66f3-4123-af6d-3b8608bb7b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:46.360259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-9af98333-66f3-4123-af6d-3b8608bb7b7d None None] MFA Rules not processed for user `c28281bb90464004a751699a49c61ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:46.361748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9af98333-66f3-4123-af6d-3b8608bb7b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:46.521777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1107/2208] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:46.526779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8b776-be5f-44b5-84fd-3dca5f2ffa95 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:46.545806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8b776-be5f-44b5-84fd-3dca5f2ffa95 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:46.555072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fdcba-eb08-4e9f-809c-680dc38e3143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:46.555643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fdcba-eb08-4e9f-809c-680dc38e3143 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:46.555927 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fdcba-eb08-4e9f-809c-680dc38e3143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:46.556213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fdcba-eb08-4e9f-809c-680dc38e3143 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:46.556849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206fdcba-eb08-4e9f-809c-680dc38e3143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:46.558966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206fdcba-eb08-4e9f-809c-680dc38e3143 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:46.686854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1102/2209] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:20:46 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:20:46.687002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af98333-66f3-4123-af6d-3b8608bb7b7d None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:46.697232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af98333-66f3-4123-af6d-3b8608bb7b7d None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:46.770879 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66523f10-68ea-4b52-ac64-b9e25c580af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:46.772618 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523f10-68ea-4b52-ac64-b9e25c580af8 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:46.786016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523f10-68ea-4b52-ac64-b9e25c580af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:46.786529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523f10-68ea-4b52-ac64-b9e25c580af8 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:46.787487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66523f10-68ea-4b52-ac64-b9e25c580af8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:46.792059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66523f10-68ea-4b52-ac64-b9e25c580af8 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:47.339724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1108/2210] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:20:46 2022] POST /identity/v3/users => generated 293 bytes in 660 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:20:47.343995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fdcba-eb08-4e9f-809c-680dc38e3143 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:47.363112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fdcba-eb08-4e9f-809c-680dc38e3143 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:47.369299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe7ae7-f0ee-4158-881b-8ce78bd0fdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:47.370289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe7ae7-f0ee-4158-881b-8ce78bd0fdd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:47.370787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe7ae7-f0ee-4158-881b-8ce78bd0fdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:47.371304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe7ae7-f0ee-4158-881b-8ce78bd0fdd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:47.372172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe7ae7-f0ee-4158-881b-8ce78bd0fdd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:47.374052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe7ae7-f0ee-4158-881b-8ce78bd0fdd5 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:47.395856 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1103/2211] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:20:47.402849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66523f10-68ea-4b52-ac64-b9e25c580af8 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:47.417121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66523f10-68ea-4b52-ac64-b9e25c580af8 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:47.426488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d76d0-e77e-4d34-9a87-f29934ddb9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:47.428533 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d76d0-e77e-4d34-9a87-f29934ddb9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:47.428533 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d76d0-e77e-4d34-9a87-f29934ddb9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:47.429060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d76d0-e77e-4d34-9a87-f29934ddb9b2 admin admin] PATH_INFO: `/v3/projects/cd71ee4e96a8476e9a5ef9f54052409c/users/ee97ab40f8254b6980508c1f7e6b72fa/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:47.452777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d76d0-e77e-4d34-9a87-f29934ddb9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd71ee4e96a8476e9a5ef9f54052409c, user_id=ee97ab40f8254b6980508c1f7e6b72fa, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:47.466318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d76d0-e77e-4d34-9a87-f29934ddb9b2 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:47.863018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1109/2212] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:20:47 2022] PUT /identity/v3/projects/cd71ee4e96a8476e9a5ef9f54052409c/users/ee97ab40f8254b6980508c1f7e6b72fa/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:47.868217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe7ae7-f0ee-4158-881b-8ce78bd0fdd5 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:47.878941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe7ae7-f0ee-4158-881b-8ce78bd0fdd5 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:47.893402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:47.937699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc347fa-e4a4-412c-9c5c-beb58b998a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:47.938318 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc347fa-e4a4-412c-9c5c-beb58b998a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:47.938934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc347fa-e4a4-412c-9c5c-beb58b998a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:47.939132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc347fa-e4a4-412c-9c5c-beb58b998a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:47.939668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc347fa-e4a4-412c-9c5c-beb58b998a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:47.941287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc347fa-e4a4-412c-9c5c-beb58b998a5b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:47.952908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1104/2213] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:20:47.956925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d76d0-e77e-4d34-9a87-f29934ddb9b2 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:47.967777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d76d0-e77e-4d34-9a87-f29934ddb9b2 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:47.972652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d6518-43e2-4a63-9e8c-8c2d2fc2458e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:47.973354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d6518-43e2-4a63-9e8c-8c2d2fc2458e admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:47.973656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d6518-43e2-4a63-9e8c-8c2d2fc2458e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:47.973931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d6518-43e2-4a63-9e8c-8c2d2fc2458e admin admin] PATH_INFO: `/v3/projects/cd71ee4e96a8476e9a5ef9f54052409c/users/ee97ab40f8254b6980508c1f7e6b72fa/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:47.976938 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d6518-43e2-4a63-9e8c-8c2d2fc2458e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd71ee4e96a8476e9a5ef9f54052409c, user_id=ee97ab40f8254b6980508c1f7e6b72fa, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:47.979319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d6518-43e2-4a63-9e8c-8c2d2fc2458e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:47.991702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1110/2214] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:20:47 2022] PUT /identity/v3/projects/cd71ee4e96a8476e9a5ef9f54052409c/users/ee97ab40f8254b6980508c1f7e6b72fa/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:53.164349 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af4b6b-284a-4bc8-97d9-3582515572f1 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:53.165079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af4b6b-284a-4bc8-97d9-3582515572f1 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:53.165457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af4b6b-284a-4bc8-97d9-3582515572f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:53.190302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-47af4b6b-284a-4bc8-97d9-3582515572f1 None None] MFA Rules not processed for user `388174c87a2245ca89a634f3b177e166`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:53.195153 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-47af4b6b-284a-4bc8-97d9-3582515572f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:53.345321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1105/2215] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:55.379968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3856f6-1ea7-4620-ad98-6b2b7038dd04 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:55.381318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3856f6-1ea7-4620-ad98-6b2b7038dd04 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:55.381318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3856f6-1ea7-4620-ad98-6b2b7038dd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:55.419312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ff3856f6-1ea7-4620-ad98-6b2b7038dd04 None None] MFA Rules not processed for user `ee97ab40f8254b6980508c1f7e6b72fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:55.421126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ff3856f6-1ea7-4620-ad98-6b2b7038dd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:55.447265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:55.543949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1111/2216] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:55.823199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af4b6b-284a-4bc8-97d9-3582515572f1 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:55.824770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-47af4b6b-284a-4bc8-97d9-3582515572f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:55.918479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af4b6b-284a-4bc8-97d9-3582515572f1 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:55.924411 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2dc4c-c7ef-47d3-afed-8fb1332c56a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:55.925086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2dc4c-c7ef-47d3-afed-8fb1332c56a5 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:55.925371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2dc4c-c7ef-47d3-afed-8fb1332c56a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:55.926434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2dc4c-c7ef-47d3-afed-8fb1332c56a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:55.992638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:55.995323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2dc4c-c7ef-47d3-afed-8fb1332c56a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:55.997094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2dc4c-c7ef-47d3-afed-8fb1332c56a5 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:56.007942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1106/2217] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:20:59.031529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa89283-64e2-4476-be71-f5b1aa93dd5d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:59.031988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa89283-64e2-4476-be71-f5b1aa93dd5d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:59.031988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa89283-64e2-4476-be71-f5b1aa93dd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:59.055348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3aa89283-64e2-4476-be71-f5b1aa93dd5d None None] MFA Rules not processed for user `c28281bb90464004a751699a49c61ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:59.056991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3aa89283-64e2-4476-be71-f5b1aa93dd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:59.064959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1112/2218] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:20:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:21:01.024302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb67e-e132-4e5b-92ea-214366501606 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:01.024781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb67e-e132-4e5b-92ea-214366501606 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:01.025135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb67e-e132-4e5b-92ea-214366501606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:01.049280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-4c8eb67e-e132-4e5b-92ea-214366501606 None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:21:01.050988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-4c8eb67e-e132-4e5b-92ea-214366501606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:21:01.129888 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1107/2219] 10.222.0.30 () {58 vars in 967 bytes} [Wed Aug 24 10:21:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:21:01.178396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa89283-64e2-4476-be71-f5b1aa93dd5d None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:21:01.255782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa89283-64e2-4476-be71-f5b1aa93dd5d None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:21:01.261793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988227c6-8e95-4bb4-832e-66c764346a05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:21:01.262416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988227c6-8e95-4bb4-832e-66c764346a05 service swift] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:01.262878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988227c6-8e95-4bb4-832e-66c764346a05 service swift] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:01.263347 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988227c6-8e95-4bb4-832e-66c764346a05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:01.276066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988227c6-8e95-4bb4-832e-66c764346a05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:21:01.277730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988227c6-8e95-4bb4-832e-66c764346a05 service swift] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:21:01.285942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1113/2220] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:21:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:21:11.113725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=27 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:21:11.282555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=54 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:21:38.480714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eb67e-e132-4e5b-92ea-214366501606 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:21:38.536540 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eb67e-e132-4e5b-92ea-214366501606 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:21:38.540187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6745d8-9b6d-4618-a999-62c1796666bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:21:38.540841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6745d8-9b6d-4618-a999-62c1796666bd service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:38.552307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6745d8-9b6d-4618-a999-62c1796666bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:38.552307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6745d8-9b6d-4618-a999-62c1796666bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:38.552307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6745d8-9b6d-4618-a999-62c1796666bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:21:38.552307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6745d8-9b6d-4618-a999-62c1796666bd service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:21:38.561662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1108/2221] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:21:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:21:43.233505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a1051-bc45-4b8c-a2ad-7444250168fa None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:43.234029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a1051-bc45-4b8c-a2ad-7444250168fa None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:43.234139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a1051-bc45-4b8c-a2ad-7444250168fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:43.268229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-413a1051-bc45-4b8c-a2ad-7444250168fa None None] MFA Rules not processed for user `71c5ac35123e425e9753f372f9c6bac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:21:43.269597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-413a1051-bc45-4b8c-a2ad-7444250168fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:21:43.332076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1114/2222] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:21:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:21:48.557447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=26 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:21:52.302272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6745d8-9b6d-4618-a999-62c1796666bd service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:21:52.304410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-cc6745d8-9b6d-4618-a999-62c1796666bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:21:52.373577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6745d8-9b6d-4618-a999-62c1796666bd service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:21:52.377540 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a9db4-e459-4141-88ea-6b2cc37d077b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:21:52.378194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a9db4-e459-4141-88ea-6b2cc37d077b service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:52.380497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a9db4-e459-4141-88ea-6b2cc37d077b service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:52.380787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a9db4-e459-4141-88ea-6b2cc37d077b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:52.382734 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9b7a9db4-e459-4141-88ea-6b2cc37d077b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:21:52.398378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a9db4-e459-4141-88ea-6b2cc37d077b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:21:52.399822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a9db4-e459-4141-88ea-6b2cc37d077b service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:21:52.413395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1109/2223] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:21:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:21:53.325269 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=25 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:21:53.515307 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a1051-bc45-4b8c-a2ad-7444250168fa None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:21:53.525229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a1051-bc45-4b8c-a2ad-7444250168fa None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:21:53.529627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e978e-58a9-4fc1-8bb6-dee4b790699a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:21:53.530353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e978e-58a9-4fc1-8bb6-dee4b790699a service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:53.530642 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e978e-58a9-4fc1-8bb6-dee4b790699a service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:53.530950 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e978e-58a9-4fc1-8bb6-dee4b790699a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:53.532900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2e4e978e-58a9-4fc1-8bb6-dee4b790699a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:21:53.547125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e978e-58a9-4fc1-8bb6-dee4b790699a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:21:53.551002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e978e-58a9-4fc1-8bb6-dee4b790699a service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:21:53.560524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1115/2224] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:21:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:01.230898 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26503645-b869-454f-9156-a3e024157414 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:01.231424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26503645-b869-454f-9156-a3e024157414 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:01.231545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26503645-b869-454f-9156-a3e024157414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:01.250843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-26503645-b869-454f-9156-a3e024157414 None None] MFA Rules not processed for user `8d72e2cc5b804b0bb556599a26d923ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:22:01.252051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-26503645-b869-454f-9156-a3e024157414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:01.309569 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1110/2225] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:22:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:22:02.569081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e978e-58a9-4fc1-8bb6-dee4b790699a service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:02.579386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e978e-58a9-4fc1-8bb6-dee4b790699a service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:02.583338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d400f-5795-4279-a803-d2dc81351c51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:02.584143 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d400f-5795-4279-a803-d2dc81351c51 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:02.584422 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d400f-5795-4279-a803-d2dc81351c51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:02.584955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d400f-5795-4279-a803-d2dc81351c51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:02.586923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-511d400f-5795-4279-a803-d2dc81351c51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:02.599974 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d400f-5795-4279-a803-d2dc81351c51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:02.601531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d400f-5795-4279-a803-d2dc81351c51 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:02.610996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1116/2226] 10.222.0.30 () {60 vars in 1396 bytes} [Wed Aug 24 10:22:02 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:05.771986 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26503645-b869-454f-9156-a3e024157414 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:05.782842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26503645-b869-454f-9156-a3e024157414 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:05.787064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef83b24-617b-49af-b968-d30e45c3576b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:05.787693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef83b24-617b-49af-b968-d30e45c3576b service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:05.787978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef83b24-617b-49af-b968-d30e45c3576b service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:05.788309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef83b24-617b-49af-b968-d30e45c3576b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:05.790209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-4ef83b24-617b-49af-b968-d30e45c3576b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:05.875736 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef83b24-617b-49af-b968-d30e45c3576b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:05.877470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef83b24-617b-49af-b968-d30e45c3576b service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:05.887717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1111/2227] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:22:12.606624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=12 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:14.887534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d400f-5795-4279-a803-d2dc81351c51 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:14.888989 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-511d400f-5795-4279-a803-d2dc81351c51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:14.900493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d400f-5795-4279-a803-d2dc81351c51 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:14.904066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6079495-d3c8-4d19-8930-9e73b21b52f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:14.904653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6079495-d3c8-4d19-8930-9e73b21b52f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:14.904976 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6079495-d3c8-4d19-8930-9e73b21b52f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:14.905254 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6079495-d3c8-4d19-8930-9e73b21b52f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:14.906947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d6079495-d3c8-4d19-8930-9e73b21b52f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:14.973156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6079495-d3c8-4d19-8930-9e73b21b52f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:14.974593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6079495-d3c8-4d19-8930-9e73b21b52f5 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:14.984473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1117/2228] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:15.883521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=79 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:18.230873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef83b24-617b-49af-b968-d30e45c3576b service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:18.242887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef83b24-617b-49af-b968-d30e45c3576b service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:18.247521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78768311-da24-4db1-9c69-2a2a43f80e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:18.248581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78768311-da24-4db1-9c69-2a2a43f80e46 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:18.249089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78768311-da24-4db1-9c69-2a2a43f80e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:18.249590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78768311-da24-4db1-9c69-2a2a43f80e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:18.252451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-78768311-da24-4db1-9c69-2a2a43f80e46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:18.284567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78768311-da24-4db1-9c69-2a2a43f80e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:18.286329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78768311-da24-4db1-9c69-2a2a43f80e46 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:18.296413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1112/2229] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:22:19.132391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6079495-d3c8-4d19-8930-9e73b21b52f5 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:19.134060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d6079495-d3c8-4d19-8930-9e73b21b52f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:19.151883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6079495-d3c8-4d19-8930-9e73b21b52f5 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:19.156905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:19.158248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:19.160995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:19.160995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] PATH_INFO: `/v3/users/388174c87a2245ca89a634f3b177e166` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:19.160995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] RBAC: Authorizing `identity:delete_user(user_id=388174c87a2245ca89a634f3b177e166)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:19.162598 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:19.333441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733546a-a810-499d-850d-1fb32c747513 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:19.333966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733546a-a810-499d-850d-1fb32c747513 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:19.333966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733546a-a810-499d-850d-1fb32c747513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:19.355195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-0733546a-a810-499d-850d-1fb32c747513 None None] MFA Rules not processed for user `c94c65ae14e94796a3b3bd0bb29a91b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:22:19.356764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0733546a-a810-499d-850d-1fb32c747513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:19.365358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1113/2230] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:22:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:22:19.426548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388174c87a2245ca89a634f3b177e166'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:19.427229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388174c87a2245ca89a634f3b177e166'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:19.438174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388174c87a2245ca89a634f3b177e166'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:19.457951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388174c87a2245ca89a634f3b177e166'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:19.539579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388174c87a2245ca89a634f3b177e166'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:19.547931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1118/2231] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:22:19 2022] DELETE /identity/v3/users/388174c87a2245ca89a634f3b177e166 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:22:19.707186 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733546a-a810-499d-850d-1fb32c747513 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:19.716876 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733546a-a810-499d-850d-1fb32c747513 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:19.768930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc79f8-1c94-4b8b-abb8-217d48bda695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:19.769536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc79f8-1c94-4b8b-abb8-217d48bda695 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:19.769814 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc79f8-1c94-4b8b-abb8-217d48bda695 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:19.770081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc79f8-1c94-4b8b-abb8-217d48bda695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:19.771787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-abfc79f8-1c94-4b8b-abb8-217d48bda695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:19.781334 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc79f8-1c94-4b8b-abb8-217d48bda695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:19.782923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc79f8-1c94-4b8b-abb8-217d48bda695 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:19.791075 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1114/2232] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:19.821863 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:19.823711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:19.903400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696a3e0-24a9-4f01-960d-2596272ae9df admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:19.908401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:19.909077 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:19.909441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:19.909711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] PATH_INFO: `/v3/projects/66a7713fb6804b4db1fbd15fa71afa74` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:19.910845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=66a7713fb6804b4db1fbd15fa71afa74)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:19.912826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:19.923426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:20.050844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66a7713fb6804b4db1fbd15fa71afa74', 'request_id': 'req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:20.058297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66a7713fb6804b4db1fbd15fa71afa74', 'request_id': 'req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:20.147759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66a7713fb6804b4db1fbd15fa71afa74', 'request_id': 'req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:20.160089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] The token cache is being invalidate because project 66a7713fb6804b4db1fbd15fa71afa74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:22:20.160613 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:20.161430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:20.162936 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1119/2233] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:22:19 2022] DELETE /identity/v3/projects/66a7713fb6804b4db1fbd15fa71afa74 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:22:20.168348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59f4b2-aece-4fce-afd2-fc38b6f8629c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:20.168675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59f4b2-aece-4fce-afd2-fc38b6f8629c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:20.169033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59f4b2-aece-4fce-afd2-fc38b6f8629c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:20.191020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-5c59f4b2-aece-4fce-afd2-fc38b6f8629c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:22:20.192232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5c59f4b2-aece-4fce-afd2-fc38b6f8629c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:20.270481 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:20.272045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1115/2234] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:22:20.272314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:20.274077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45b066-d74d-4119-8d18-820701de9d11 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:20.274367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45b066-d74d-4119-8d18-820701de9d11 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:20.274673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45b066-d74d-4119-8d18-820701de9d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:20.300444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-bc45b066-d74d-4119-8d18-820701de9d11 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:22:20.301734 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-bc45b066-d74d-4119-8d18-820701de9d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:20.316839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1116/2235] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:22:20.318101 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45b066-d74d-4119-8d18-820701de9d11 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:20.328460 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45b066-d74d-4119-8d18-820701de9d11 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:20.333409 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d0fd7-2d8b-44be-ad82-d8bb331ad2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:20.334014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d0fd7-2d8b-44be-ad82-d8bb331ad2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:20.334599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d0fd7-2d8b-44be-ad82-d8bb331ad2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:20.334599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d0fd7-2d8b-44be-ad82-d8bb331ad2e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:20.335195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d0fd7-2d8b-44be-ad82-d8bb331ad2e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:22:20.335504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d0fd7-2d8b-44be-ad82-d8bb331ad2e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:20.337409 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d0fd7-2d8b-44be-ad82-d8bb331ad2e9 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:20.351158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1117/2236] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:22:20.357867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d0fd7-2d8b-44be-ad82-d8bb331ad2e9 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:20.368636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d0fd7-2d8b-44be-ad82-d8bb331ad2e9 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:20.373403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98373a35-244b-4efa-bb95-82f0f5abec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:20.374133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98373a35-244b-4efa-bb95-82f0f5abec51 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:20.374351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98373a35-244b-4efa-bb95-82f0f5abec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:20.374657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98373a35-244b-4efa-bb95-82f0f5abec51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:20.375286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98373a35-244b-4efa-bb95-82f0f5abec51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:20.377972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98373a35-244b-4efa-bb95-82f0f5abec51 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:20.382897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2f9f2-52cb-43cc-8543-36ee475b7d5b admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:20.388183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764a2ad-1aaa-4f71-8407-f247d95118bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:20.388789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764a2ad-1aaa-4f71-8407-f247d95118bb service placement] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:20.389105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764a2ad-1aaa-4f71-8407-f247d95118bb service placement] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:20.389380 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764a2ad-1aaa-4f71-8407-f247d95118bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:20.391261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2764a2ad-1aaa-4f71-8407-f247d95118bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:20.403941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764a2ad-1aaa-4f71-8407-f247d95118bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:20.405626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764a2ad-1aaa-4f71-8407-f247d95118bb service placement] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:20.416681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1120/2237] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:20.491434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1118/2238] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:22:20 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:22:20.494076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764a2ad-1aaa-4f71-8407-f247d95118bb service placement] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:20.508148 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764a2ad-1aaa-4f71-8407-f247d95118bb service placement] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:20.576115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca27df-39bd-461f-8076-86d63468fd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:20.576764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca27df-39bd-461f-8076-86d63468fd74 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:20.577064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca27df-39bd-461f-8076-86d63468fd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:20.577338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca27df-39bd-461f-8076-86d63468fd74 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:20.577990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca27df-39bd-461f-8076-86d63468fd74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:20.580803 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca27df-39bd-461f-8076-86d63468fd74 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:20.709103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1121/2239] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:22:20 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:22:20.713163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98373a35-244b-4efa-bb95-82f0f5abec51 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:20.723958 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98373a35-244b-4efa-bb95-82f0f5abec51 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:20.729001 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2473f3f-4d3a-463e-8384-fa94d95f2abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:20.729717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2473f3f-4d3a-463e-8384-fa94d95f2abc admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:20.730045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2473f3f-4d3a-463e-8384-fa94d95f2abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:20.730389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2473f3f-4d3a-463e-8384-fa94d95f2abc admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:20.731110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2473f3f-4d3a-463e-8384-fa94d95f2abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:20.732781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2473f3f-4d3a-463e-8384-fa94d95f2abc admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:20.743525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1119/2240] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:22:20.747783 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca27df-39bd-461f-8076-86d63468fd74 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:20.759252 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca27df-39bd-461f-8076-86d63468fd74 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:20.763915 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da51c65-8b2c-4aad-8efe-271de261e54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:20.764510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da51c65-8b2c-4aad-8efe-271de261e54b admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:20.764808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da51c65-8b2c-4aad-8efe-271de261e54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:20.765101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da51c65-8b2c-4aad-8efe-271de261e54b admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:20.765738 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da51c65-8b2c-4aad-8efe-271de261e54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:20.769043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da51c65-8b2c-4aad-8efe-271de261e54b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:20.778760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1122/2241] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:22:20.783115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2473f3f-4d3a-463e-8384-fa94d95f2abc admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:20.793857 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2473f3f-4d3a-463e-8384-fa94d95f2abc admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:20.800807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f713c2-32a8-46ad-bf08-2af50ecbf691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:20.801504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f713c2-32a8-46ad-bf08-2af50ecbf691 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:20.801785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f713c2-32a8-46ad-bf08-2af50ecbf691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:20.802263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f713c2-32a8-46ad-bf08-2af50ecbf691 admin admin] PATH_INFO: `/v3/projects/0dc25f4988ee446686b0e12a0b54892a/users/ea449a6088aa490f879f2de802de134d/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:20.821143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f713c2-32a8-46ad-bf08-2af50ecbf691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc25f4988ee446686b0e12a0b54892a, user_id=ea449a6088aa490f879f2de802de134d, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:20.824705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f713c2-32a8-46ad-bf08-2af50ecbf691 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:20.906206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1120/2242] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:22:20 2022] PUT /identity/v3/projects/0dc25f4988ee446686b0e12a0b54892a/users/ea449a6088aa490f879f2de802de134d/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:22:20.908883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c5e26-7bdc-4b72-83f0-91e371e77785 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:20.909186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c5e26-7bdc-4b72-83f0-91e371e77785 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:20.909517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c5e26-7bdc-4b72-83f0-91e371e77785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:20.917466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:20.940087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-a38c5e26-7bdc-4b72-83f0-91e371e77785 None None] MFA Rules not processed for user `ea449a6088aa490f879f2de802de134d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:22:20.941355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a38c5e26-7bdc-4b72-83f0-91e371e77785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:21.029927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1123/2243] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:22:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:22:21.041640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f713c2-32a8-46ad-bf08-2af50ecbf691 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:21.044031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b2f713c2-32a8-46ad-bf08-2af50ecbf691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:21.061281 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:21.135753 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f713c2-32a8-46ad-bf08-2af50ecbf691 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:21.143262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e85d9a-46a5-441f-b56b-e453de09ddba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:21.143698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e85d9a-46a5-441f-b56b-e453de09ddba service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:21.143984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e85d9a-46a5-441f-b56b-e453de09ddba service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:21.144265 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e85d9a-46a5-441f-b56b-e453de09ddba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:21.152885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e85d9a-46a5-441f-b56b-e453de09ddba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:21.154513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e85d9a-46a5-441f-b56b-e453de09ddba service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:21.163797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1121/2244] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:27.374157 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c5e26-7bdc-4b72-83f0-91e371e77785 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:27.379265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a38c5e26-7bdc-4b72-83f0-91e371e77785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:27.392622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c5e26-7bdc-4b72-83f0-91e371e77785 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:27.398526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da03f48-6a80-485c-aba5-af08903280e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:27.399144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da03f48-6a80-485c-aba5-af08903280e9 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:27.399428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da03f48-6a80-485c-aba5-af08903280e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:27.399704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da03f48-6a80-485c-aba5-af08903280e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:27.401557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-1da03f48-6a80-485c-aba5-af08903280e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:27.414903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da03f48-6a80-485c-aba5-af08903280e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:27.416505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da03f48-6a80-485c-aba5-af08903280e9 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:27.428130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1124/2245] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:30.578443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e85d9a-46a5-441f-b56b-e453de09ddba service nova] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:30.639854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e85d9a-46a5-441f-b56b-e453de09ddba service nova] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:30.644634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:30.645293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:30.645578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:30.645854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] PATH_INFO: `/v3/users/ea449a6088aa490f879f2de802de134d` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:30.646970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea449a6088aa490f879f2de802de134d)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:30.648907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:30.847292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea449a6088aa490f879f2de802de134d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:30.847945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea449a6088aa490f879f2de802de134d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:30.854835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea449a6088aa490f879f2de802de134d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:30.859966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea449a6088aa490f879f2de802de134d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:30.925104 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea449a6088aa490f879f2de802de134d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:30.945081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1122/2246] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:22:30 2022] DELETE /identity/v3/users/ea449a6088aa490f879f2de802de134d => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:22:30.949867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24995b79-0337-44f5-9cf5-a61d3e72487c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:30.950330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24995b79-0337-44f5-9cf5-a61d3e72487c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:30.953246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24995b79-0337-44f5-9cf5-a61d3e72487c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:30.969227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-24995b79-0337-44f5-9cf5-a61d3e72487c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:22:30.970519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-24995b79-0337-44f5-9cf5-a61d3e72487c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:31.031872 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1125/2247] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:22:31.042526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:31.043932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:31.058944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:31.107833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa0e05-cf55-4de4-994a-59ed723a2dd5 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:31.108441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7665fa-3f85-47c6-b980-9916be7231dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:31.108709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7665fa-3f85-47c6-b980-9916be7231dd service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:31.109033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7665fa-3f85-47c6-b980-9916be7231dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:31.109305 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7665fa-3f85-47c6-b980-9916be7231dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:31.119634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7665fa-3f85-47c6-b980-9916be7231dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:31.121117 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7665fa-3f85-47c6-b980-9916be7231dd service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:31.131956 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1123/2248] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:22:31.590278 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24995b79-0337-44f5-9cf5-a61d3e72487c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:31.592132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-24995b79-0337-44f5-9cf5-a61d3e72487c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:31.611067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24995b79-0337-44f5-9cf5-a61d3e72487c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:31.619999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:31.620999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:31.621568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:31.621936 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] PATH_INFO: `/v3/projects/0dc25f4988ee446686b0e12a0b54892a` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:31.623332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc25f4988ee446686b0e12a0b54892a)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:31.625532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:31.769297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc25f4988ee446686b0e12a0b54892a', 'request_id': 'req-d4af4108-6862-4b74-bb3a-a2cbc996313f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:31.774724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc25f4988ee446686b0e12a0b54892a', 'request_id': 'req-d4af4108-6862-4b74-bb3a-a2cbc996313f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:31.841781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc25f4988ee446686b0e12a0b54892a', 'request_id': 'req-d4af4108-6862-4b74-bb3a-a2cbc996313f'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:31.855876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] The token cache is being invalidate because project 0dc25f4988ee446686b0e12a0b54892a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:22:31.856172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:31.856819 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d4af4108-6862-4b74-bb3a-a2cbc996313f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:22:31.858660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1126/2249] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:22:31 2022] DELETE /identity/v3/projects/0dc25f4988ee446686b0e12a0b54892a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:22:31.864154 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2b8a-5837-4249-b583-1050dc079537 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:31.864644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2b8a-5837-4249-b583-1050dc079537 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:31.864759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2b8a-5837-4249-b583-1050dc079537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:31.899014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e29b2b8a-5837-4249-b583-1050dc079537 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:22:31.900985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e29b2b8a-5837-4249-b583-1050dc079537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:31.974573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1124/2250] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:22:31.983398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44574-766d-4d5a-93bd-dcb3aecaab3b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:31.983944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44574-766d-4d5a-93bd-dcb3aecaab3b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:31.983944 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44574-766d-4d5a-93bd-dcb3aecaab3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:31.995064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:32.010043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-97f44574-766d-4d5a-93bd-dcb3aecaab3b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:22:32.011863 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-97f44574-766d-4d5a-93bd-dcb3aecaab3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:32.027298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1127/2251] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:22:32.031713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b2b8a-5837-4249-b583-1050dc079537 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:32.041959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b2b8a-5837-4249-b583-1050dc079537 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:32.046676 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b224be9-dd48-4370-9169-f8ef173a79f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:32.047282 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b224be9-dd48-4370-9169-f8ef173a79f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:32.047558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b224be9-dd48-4370-9169-f8ef173a79f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:32.048193 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b224be9-dd48-4370-9169-f8ef173a79f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:32.048488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b224be9-dd48-4370-9169-f8ef173a79f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:22:32.048791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b224be9-dd48-4370-9169-f8ef173a79f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:32.050786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b224be9-dd48-4370-9169-f8ef173a79f6 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:32.063569 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1125/2252] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:22:32.068561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f44574-766d-4d5a-93bd-dcb3aecaab3b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:32.080382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f44574-766d-4d5a-93bd-dcb3aecaab3b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:32.085680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a31f15-75ea-4e2e-93c4-7ec2aef036e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:32.086227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a31f15-75ea-4e2e-93c4-7ec2aef036e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:32.086506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a31f15-75ea-4e2e-93c4-7ec2aef036e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:32.086777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a31f15-75ea-4e2e-93c4-7ec2aef036e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:32.087414 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a31f15-75ea-4e2e-93c4-7ec2aef036e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:32.089912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a31f15-75ea-4e2e-93c4-7ec2aef036e6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:32.179229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1128/2253] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:22:32 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:22:32.183469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b224be9-dd48-4370-9169-f8ef173a79f6 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:32.195144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b224be9-dd48-4370-9169-f8ef173a79f6 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:32.254635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc7ed1-093c-43e9-8112-cee03b36d366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:32.255254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc7ed1-093c-43e9-8112-cee03b36d366 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:32.255532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc7ed1-093c-43e9-8112-cee03b36d366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:32.255848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc7ed1-093c-43e9-8112-cee03b36d366 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:32.256529 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc7ed1-093c-43e9-8112-cee03b36d366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:32.258491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc7ed1-093c-43e9-8112-cee03b36d366 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:32.354745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1126/2254] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:22:32 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:22:32.355682 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a31f15-75ea-4e2e-93c4-7ec2aef036e6 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:32.368280 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a31f15-75ea-4e2e-93c4-7ec2aef036e6 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:32.373089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74e47-f804-4647-81d7-ed556a68d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:32.374032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74e47-f804-4647-81d7-ed556a68d389 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:32.374032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74e47-f804-4647-81d7-ed556a68d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:32.376834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74e47-f804-4647-81d7-ed556a68d389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:32.376834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c74e47-f804-4647-81d7-ed556a68d389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:32.376834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c74e47-f804-4647-81d7-ed556a68d389 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:32.388086 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1129/2255] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:22:32.392311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc7ed1-093c-43e9-8112-cee03b36d366 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:32.402676 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc7ed1-093c-43e9-8112-cee03b36d366 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:32.407538 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fe3de-3bc2-4630-bf74-eed100e78ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:32.408268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fe3de-3bc2-4630-bf74-eed100e78ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:32.408547 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fe3de-3bc2-4630-bf74-eed100e78ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:32.408824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fe3de-3bc2-4630-bf74-eed100e78ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:32.409502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fe3de-3bc2-4630-bf74-eed100e78ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:32.411088 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fe3de-3bc2-4630-bf74-eed100e78ac2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:32.421580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1127/2256] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:22:32.426595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74e47-f804-4647-81d7-ed556a68d389 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:32.436561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74e47-f804-4647-81d7-ed556a68d389 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:32.442013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff970294-78bf-475f-b2f4-2866075fbc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:32.442650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff970294-78bf-475f-b2f4-2866075fbc48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:32.442930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff970294-78bf-475f-b2f4-2866075fbc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:32.443205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff970294-78bf-475f-b2f4-2866075fbc48 admin admin] PATH_INFO: `/v3/projects/40b115233de14f5d96c0da76fc845c53/users/d0c1c0a3e1254ceab08c0a6032a50ef1/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:32.458723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff970294-78bf-475f-b2f4-2866075fbc48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b115233de14f5d96c0da76fc845c53, user_id=d0c1c0a3e1254ceab08c0a6032a50ef1, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:32.461442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff970294-78bf-475f-b2f4-2866075fbc48 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:32.541931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1130/2257] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:22:32 2022] PUT /identity/v3/projects/40b115233de14f5d96c0da76fc845c53/users/d0c1c0a3e1254ceab08c0a6032a50ef1/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:22:32.555210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ee53-4909-4743-aa71-eb0730d23d7a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:32.555498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ee53-4909-4743-aa71-eb0730d23d7a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:32.555770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ee53-4909-4743-aa71-eb0730d23d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:32.582844 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c2f1ee53-4909-4743-aa71-eb0730d23d7a None None] MFA Rules not processed for user `d0c1c0a3e1254ceab08c0a6032a50ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:22:32.584044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c2f1ee53-4909-4743-aa71-eb0730d23d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:32.592055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:32.651362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1128/2258] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:22:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:22:32.655343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff970294-78bf-475f-b2f4-2866075fbc48 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:32.717102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff970294-78bf-475f-b2f4-2866075fbc48 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:32.722371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fa498-aadc-4296-b9a5-c453ba4db314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:32.722972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa498-aadc-4296-b9a5-c453ba4db314 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:32.723257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa498-aadc-4296-b9a5-c453ba4db314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:32.723531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa498-aadc-4296-b9a5-c453ba4db314 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:32.724147 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fa498-aadc-4296-b9a5-c453ba4db314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:32.726080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fa498-aadc-4296-b9a5-c453ba4db314 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:32.818211 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1131/2259] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:22:32 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:22:32.822343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1ee53-4909-4743-aa71-eb0730d23d7a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:32.833859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1ee53-4909-4743-aa71-eb0730d23d7a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:32.892054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57078a7-420f-4500-baa6-d325f5afac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:32.892757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57078a7-420f-4500-baa6-d325f5afac27 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:32.892977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57078a7-420f-4500-baa6-d325f5afac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:32.893514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57078a7-420f-4500-baa6-d325f5afac27 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:32.894132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57078a7-420f-4500-baa6-d325f5afac27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:32.896033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57078a7-420f-4500-baa6-d325f5afac27 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:32.965079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1129/2260] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:22:32 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:22:32.969196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fa498-aadc-4296-b9a5-c453ba4db314 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:32.979723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fa498-aadc-4296-b9a5-c453ba4db314 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:32.984696 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24927cc9-8c6a-4eee-88d3-0d6f0d35b3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:32.985334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24927cc9-8c6a-4eee-88d3-0d6f0d35b3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:32.985622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24927cc9-8c6a-4eee-88d3-0d6f0d35b3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:32.986590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24927cc9-8c6a-4eee-88d3-0d6f0d35b3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:32.987537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24927cc9-8c6a-4eee-88d3-0d6f0d35b3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:32.989272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24927cc9-8c6a-4eee-88d3-0d6f0d35b3d2 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:33.002654 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1132/2261] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:22:33.007019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57078a7-420f-4500-baa6-d325f5afac27 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:33.017705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57078a7-420f-4500-baa6-d325f5afac27 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:33.022516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97fa16-b376-4d9f-9fb0-9b00cb51e3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:33.023217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97fa16-b376-4d9f-9fb0-9b00cb51e3b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:33.023503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97fa16-b376-4d9f-9fb0-9b00cb51e3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:33.024094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97fa16-b376-4d9f-9fb0-9b00cb51e3b2 admin admin] PATH_INFO: `/v3/projects/2f519061fc3c49cdaa1a30607d05af2e/users/08945aa391af46dd9a867dde9b5432c8/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:33.039412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97fa16-b376-4d9f-9fb0-9b00cb51e3b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f519061fc3c49cdaa1a30607d05af2e, user_id=08945aa391af46dd9a867dde9b5432c8, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:33.042230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97fa16-b376-4d9f-9fb0-9b00cb51e3b2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:33.111482 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1130/2262] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:22:33 2022] PUT /identity/v3/projects/2f519061fc3c49cdaa1a30607d05af2e/users/08945aa391af46dd9a867dde9b5432c8/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:22:33.116081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24927cc9-8c6a-4eee-88d3-0d6f0d35b3d2 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:33.126318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24927cc9-8c6a-4eee-88d3-0d6f0d35b3d2 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:33.192835 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cfcc9-766f-4c81-93dc-99cd5887fa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:33.193447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cfcc9-766f-4c81-93dc-99cd5887fa21 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:33.193734 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cfcc9-766f-4c81-93dc-99cd5887fa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:33.194095 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cfcc9-766f-4c81-93dc-99cd5887fa21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:33.194738 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cfcc9-766f-4c81-93dc-99cd5887fa21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:33.196613 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cfcc9-766f-4c81-93dc-99cd5887fa21 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:33.206516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1133/2263] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:22:33.210700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97fa16-b376-4d9f-9fb0-9b00cb51e3b2 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:33.221173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97fa16-b376-4d9f-9fb0-9b00cb51e3b2 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:33.225752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232e56a-e23a-49ed-b84f-e120c47abe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:33.226390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e56a-e23a-49ed-b84f-e120c47abe7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:33.226671 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e56a-e23a-49ed-b84f-e120c47abe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:33.226948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e56a-e23a-49ed-b84f-e120c47abe7f admin admin] PATH_INFO: `/v3/projects/2f519061fc3c49cdaa1a30607d05af2e/users/08945aa391af46dd9a867dde9b5432c8/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:33.228829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232e56a-e23a-49ed-b84f-e120c47abe7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f519061fc3c49cdaa1a30607d05af2e, user_id=08945aa391af46dd9a867dde9b5432c8, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:33.231484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232e56a-e23a-49ed-b84f-e120c47abe7f admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:33.242516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1131/2264] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:22:33 2022] PUT /identity/v3/projects/2f519061fc3c49cdaa1a30607d05af2e/users/08945aa391af46dd9a867dde9b5432c8/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:22:33.253448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37142d-e613-4618-bcf4-1c0fada37269 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:33.253781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37142d-e613-4618-bcf4-1c0fada37269 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:33.254055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37142d-e613-4618-bcf4-1c0fada37269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:33.274573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:33.282075 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3e37142d-e613-4618-bcf4-1c0fada37269 None None] MFA Rules not processed for user `08945aa391af46dd9a867dde9b5432c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:22:33.285958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3e37142d-e613-4618-bcf4-1c0fada37269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:33.355472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1134/2265] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:22:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:22:33.369943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232e56a-e23a-49ed-b84f-e120c47abe7f admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:33.371636 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1232e56a-e23a-49ed-b84f-e120c47abe7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:33.433962 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232e56a-e23a-49ed-b84f-e120c47abe7f admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:33.440502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdbc8a-85bd-4756-b429-6249f150ba8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:33.440502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdbc8a-85bd-4756-b429-6249f150ba8a service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:33.440502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdbc8a-85bd-4756-b429-6249f150ba8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:33.440502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdbc8a-85bd-4756-b429-6249f150ba8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:33.448046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdbc8a-85bd-4756-b429-6249f150ba8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:33.449460 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdbc8a-85bd-4756-b429-6249f150ba8a service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:33.459123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1132/2266] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:33.668205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37142d-e613-4618-bcf4-1c0fada37269 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:33.678207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37142d-e613-4618-bcf4-1c0fada37269 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:33.681795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7f632-5bdd-43ba-a4a1-2b91061c24f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:33.682384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7f632-5bdd-43ba-a4a1-2b91061c24f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:33.682667 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7f632-5bdd-43ba-a4a1-2b91061c24f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:33.682942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7f632-5bdd-43ba-a4a1-2b91061c24f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:33.739605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7f632-5bdd-43ba-a4a1-2b91061c24f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:33.740987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7f632-5bdd-43ba-a4a1-2b91061c24f0 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:33.749792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1135/2267] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:41.919326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdbc8a-85bd-4756-b429-6249f150ba8a service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:41.920704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-63bdbc8a-85bd-4756-b429-6249f150ba8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:41.927975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:41.993683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdbc8a-85bd-4756-b429-6249f150ba8a service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:41.999164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f3aa1-5a09-4465-8ea0-beecd0de0864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:41.999772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f3aa1-5a09-4465-8ea0-beecd0de0864 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:42.000058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f3aa1-5a09-4465-8ea0-beecd0de0864 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:42.000317 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f3aa1-5a09-4465-8ea0-beecd0de0864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:42.002207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0b9f3aa1-5a09-4465-8ea0-beecd0de0864 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:42.064931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f3aa1-5a09-4465-8ea0-beecd0de0864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:42.067355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f3aa1-5a09-4465-8ea0-beecd0de0864 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:42.076450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1133/2268] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:43.745799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=57 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:52.072474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=54 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:19.761350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7f632-5bdd-43ba-a4a1-2b91061c24f0 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:19.774223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7f632-5bdd-43ba-a4a1-2b91061c24f0 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:19.779696 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4abbd-3a41-4e05-a3dd-86af1af655db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:19.780297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4abbd-3a41-4e05-a3dd-86af1af655db service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:19.780575 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4abbd-3a41-4e05-a3dd-86af1af655db service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:19.780849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4abbd-3a41-4e05-a3dd-86af1af655db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:19.782596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-19f4abbd-3a41-4e05-a3dd-86af1af655db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:19.795047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4abbd-3a41-4e05-a3dd-86af1af655db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:19.796640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4abbd-3a41-4e05-a3dd-86af1af655db service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:19.807922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1136/2269] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:20.995393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d4c8c-4fba-4921-97a1-e9dea4aeded6 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:20.995941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d4c8c-4fba-4921-97a1-e9dea4aeded6 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:20.995941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d4c8c-4fba-4921-97a1-e9dea4aeded6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:21.018104 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-412d4c8c-4fba-4921-97a1-e9dea4aeded6 None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:21.019772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-412d4c8c-4fba-4921-97a1-e9dea4aeded6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:21.083136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1134/2270] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:23:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:21.366513 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4abbd-3a41-4e05-a3dd-86af1af655db service nova] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:21.368136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-19f4abbd-3a41-4e05-a3dd-86af1af655db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:21.449028 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4abbd-3a41-4e05-a3dd-86af1af655db service nova] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:21.455119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576aef31-537e-4b9e-9f84-88d0e43e348e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:21.455959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aef31-537e-4b9e-9f84-88d0e43e348e service swift] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:21.456335 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aef31-537e-4b9e-9f84-88d0e43e348e service swift] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:21.456813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aef31-537e-4b9e-9f84-88d0e43e348e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:21.469301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576aef31-537e-4b9e-9f84-88d0e43e348e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:21.471000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576aef31-537e-4b9e-9f84-88d0e43e348e service swift] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:21.487943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1137/2271] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:23:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:23:26.460932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d4c8c-4fba-4921-97a1-e9dea4aeded6 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:26.561489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d4c8c-4fba-4921-97a1-e9dea4aeded6 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:26.566743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:26.567608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:26.568102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:26.568638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] PATH_INFO: `/v3/users/d0c1c0a3e1254ceab08c0a6032a50ef1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:26.570353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0c1c0a3e1254ceab08c0a6032a50ef1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:26.577077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:26.835739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0c1c0a3e1254ceab08c0a6032a50ef1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:26.836658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0c1c0a3e1254ceab08c0a6032a50ef1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:26.847483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0c1c0a3e1254ceab08c0a6032a50ef1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:26.848301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0c1c0a3e1254ceab08c0a6032a50ef1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:26.955164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0c1c0a3e1254ceab08c0a6032a50ef1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:26.993616 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1135/2272] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:26 2022] DELETE /identity/v3/users/d0c1c0a3e1254ceab08c0a6032a50ef1 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:26.998178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaadc4b-fae6-4fb3-bd0b-895611f93b3a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:26.999951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaadc4b-fae6-4fb3-bd0b-895611f93b3a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:27.001505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaadc4b-fae6-4fb3-bd0b-895611f93b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:27.030671 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-2eaadc4b-fae6-4fb3-bd0b-895611f93b3a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:27.032262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2eaadc4b-fae6-4fb3-bd0b-895611f93b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:27.124369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1138/2273] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:27.136393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:27.139304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:27.180999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:27.219605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e179697-59c2-4729-adfc-866056eee1ea admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:27.223280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679b850-b4a8-40c1-845a-76bf9913ab06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:27.223891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679b850-b4a8-40c1-845a-76bf9913ab06 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:27.224179 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679b850-b4a8-40c1-845a-76bf9913ab06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:27.224479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679b850-b4a8-40c1-845a-76bf9913ab06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:27.238311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679b850-b4a8-40c1-845a-76bf9913ab06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:27.239832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679b850-b4a8-40c1-845a-76bf9913ab06 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:27.253523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1136/2274] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:28.246154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaadc4b-fae6-4fb3-bd0b-895611f93b3a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:28.248162 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2eaadc4b-fae6-4fb3-bd0b-895611f93b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:28.260542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaadc4b-fae6-4fb3-bd0b-895611f93b3a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:28.299695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:28.299695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:28.299695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:28.299695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] PATH_INFO: `/v3/projects/40b115233de14f5d96c0da76fc845c53` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:28.300533 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40b115233de14f5d96c0da76fc845c53)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:28.300533 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:28.495700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40b115233de14f5d96c0da76fc845c53', 'request_id': 'req-3712a265-cabc-42a4-b19a-c7ba12e70f58'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:28.507038 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40b115233de14f5d96c0da76fc845c53', 'request_id': 'req-3712a265-cabc-42a4-b19a-c7ba12e70f58'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:28.617899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40b115233de14f5d96c0da76fc845c53', 'request_id': 'req-3712a265-cabc-42a4-b19a-c7ba12e70f58'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:28.627505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:28.644452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] The token cache is being invalidate because project 40b115233de14f5d96c0da76fc845c53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:28.644452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:28.644452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:28.644452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1139/2275] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:28 2022] DELETE /identity/v3/projects/40b115233de14f5d96c0da76fc845c53 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:28.645044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679b850-b4a8-40c1-845a-76bf9913ab06 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:28.655233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679b850-b4a8-40c1-845a-76bf9913ab06 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:28.727791 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:28.729037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:28.729037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:28.729415 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] PATH_INFO: `/v3/users/08945aa391af46dd9a867dde9b5432c8` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:28.730739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08945aa391af46dd9a867dde9b5432c8)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:28.733123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:29.012665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08945aa391af46dd9a867dde9b5432c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:29.013398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08945aa391af46dd9a867dde9b5432c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:29.023843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08945aa391af46dd9a867dde9b5432c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:29.025000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08945aa391af46dd9a867dde9b5432c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:29.103099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08945aa391af46dd9a867dde9b5432c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:29.127908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1137/2276] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:28 2022] DELETE /identity/v3/users/08945aa391af46dd9a867dde9b5432c8 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:29.651409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:29.653180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:29.730275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:29.732686 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:29.766518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712a265-cabc-42a4-b19a-c7ba12e70f58 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:29.835673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5829a-12c3-4fed-b0f1-66e466798d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:29.836597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5829a-12c3-4fed-b0f1-66e466798d59 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:29.836732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:29.837189 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5829a-12c3-4fed-b0f1-66e466798d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:29.837660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5829a-12c3-4fed-b0f1-66e466798d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:29.839646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2be5829a-12c3-4fed-b0f1-66e466798d59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:29.860925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e4f42-09cc-4041-a8a5-0d6f2430e1c3 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:29.868814 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:29.871515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:29.871515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:29.871515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] PATH_INFO: `/v3/projects/2f519061fc3c49cdaa1a30607d05af2e` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:29.871898 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f519061fc3c49cdaa1a30607d05af2e)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:29.874201 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:29.935425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5829a-12c3-4fed-b0f1-66e466798d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:29.938215 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5829a-12c3-4fed-b0f1-66e466798d59 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:29.951442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1140/2277] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:30.093987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f519061fc3c49cdaa1a30607d05af2e', 'request_id': 'req-53255d9d-d566-4b3d-96d6-3ec401fbd18a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:30.111200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f519061fc3c49cdaa1a30607d05af2e', 'request_id': 'req-53255d9d-d566-4b3d-96d6-3ec401fbd18a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:30.119672 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f519061fc3c49cdaa1a30607d05af2e', 'request_id': 'req-53255d9d-d566-4b3d-96d6-3ec401fbd18a'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:30.276640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] The token cache is being invalidate because project 2f519061fc3c49cdaa1a30607d05af2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:30.277023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:30.281291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-53255d9d-d566-4b3d-96d6-3ec401fbd18a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:30.284517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1138/2278] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:29 2022] DELETE /identity/v3/projects/2f519061fc3c49cdaa1a30607d05af2e => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:30.290357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e02f-562d-4f55-af1e-07ec9b607342 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:30.290649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e02f-562d-4f55-af1e-07ec9b607342 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:30.290929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e02f-562d-4f55-af1e-07ec9b607342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:30.324487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-40a8e02f-562d-4f55-af1e-07ec9b607342 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:30.325878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-40a8e02f-562d-4f55-af1e-07ec9b607342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:30.424467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1141/2279] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:30.434083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cfbfb-12f6-4151-b898-8114ced9da47 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:30.434643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cfbfb-12f6-4151-b898-8114ced9da47 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:30.434926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cfbfb-12f6-4151-b898-8114ced9da47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:30.471030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1f9cfbfb-12f6-4151-b898-8114ced9da47 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:30.472279 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1f9cfbfb-12f6-4151-b898-8114ced9da47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:30.495124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8e02f-562d-4f55-af1e-07ec9b607342 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:30.501658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1139/2280] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:30.505659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8e02f-562d-4f55-af1e-07ec9b607342 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:30.511413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346639a-4f91-4699-8433-94567277dc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:30.512096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346639a-4f91-4699-8433-94567277dc76 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:30.512444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346639a-4f91-4699-8433-94567277dc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:30.512781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346639a-4f91-4699-8433-94567277dc76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:30.513669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346639a-4f91-4699-8433-94567277dc76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:23:30.514052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346639a-4f91-4699-8433-94567277dc76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:30.515751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346639a-4f91-4699-8433-94567277dc76 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:30.517768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:30.530968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1142/2281] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:23:30.535517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cfbfb-12f6-4151-b898-8114ced9da47 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:30.546163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cfbfb-12f6-4151-b898-8114ced9da47 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:30.551348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199478d9-c83d-4474-8d21-04f673c0d492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:30.551966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199478d9-c83d-4474-8d21-04f673c0d492 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:30.552252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199478d9-c83d-4474-8d21-04f673c0d492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:30.552527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199478d9-c83d-4474-8d21-04f673c0d492 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:30.553854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199478d9-c83d-4474-8d21-04f673c0d492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:30.564459 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199478d9-c83d-4474-8d21-04f673c0d492 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:30.565452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346639a-4f91-4699-8433-94567277dc76 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:30.567449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8346639a-4f91-4699-8433-94567277dc76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:30.667748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1140/2282] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:30 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:30.669052 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199478d9-c83d-4474-8d21-04f673c0d492 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:30.701564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346639a-4f91-4699-8433-94567277dc76 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:30.781768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a553f-acde-4e86-b69f-2201ca1d38d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:30.782822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a553f-acde-4e86-b69f-2201ca1d38d3 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:30.783153 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a553f-acde-4e86-b69f-2201ca1d38d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:30.783607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a553f-acde-4e86-b69f-2201ca1d38d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:30.788908 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-eb7a553f-acde-4e86-b69f-2201ca1d38d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:30.856076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199478d9-c83d-4474-8d21-04f673c0d492 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:30.859422 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a553f-acde-4e86-b69f-2201ca1d38d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:30.861267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a553f-acde-4e86-b69f-2201ca1d38d3 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:30.864077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef792803-8e89-4904-9e84-cb62808acf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:30.865007 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792803-8e89-4904-9e84-cb62808acf8e admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:30.865777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792803-8e89-4904-9e84-cb62808acf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:30.866316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792803-8e89-4904-9e84-cb62808acf8e admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:30.867162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef792803-8e89-4904-9e84-cb62808acf8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:30.869931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef792803-8e89-4904-9e84-cb62808acf8e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:30.872784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1143/2283] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:23:31.002020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1141/2284] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:30 2022] POST /identity/v3/users => generated 285 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:31.006976 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a553f-acde-4e86-b69f-2201ca1d38d3 service nova] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:31.019605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a553f-acde-4e86-b69f-2201ca1d38d3 service nova] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:31.028324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f8882-0ebb-44a2-842e-62d1c4d3c14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:31.029079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f8882-0ebb-44a2-842e-62d1c4d3c14e admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:31.029425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f8882-0ebb-44a2-842e-62d1c4d3c14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:31.029839 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f8882-0ebb-44a2-842e-62d1c4d3c14e admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:31.030565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f8882-0ebb-44a2-842e-62d1c4d3c14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:31.033085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f8882-0ebb-44a2-842e-62d1c4d3c14e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:31.051064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef792803-8e89-4904-9e84-cb62808acf8e admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:31.051646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1144/2285] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:31.070776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef792803-8e89-4904-9e84-cb62808acf8e admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:31.076802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5be25-e618-4260-bfb7-652163632976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:31.077483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5be25-e618-4260-bfb7-652163632976 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:31.077766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5be25-e618-4260-bfb7-652163632976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:31.078098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5be25-e618-4260-bfb7-652163632976 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:31.078811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5be25-e618-4260-bfb7-652163632976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:31.080493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5be25-e618-4260-bfb7-652163632976 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:31.101349 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f8882-0ebb-44a2-842e-62d1c4d3c14e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:31.108848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1142/2286] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:31.113540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f8882-0ebb-44a2-842e-62d1c4d3c14e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:31.119061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37a5dd-6c03-4461-9b2a-34d4f66001bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:31.119822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37a5dd-6c03-4461-9b2a-34d4f66001bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:31.120134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37a5dd-6c03-4461-9b2a-34d4f66001bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:31.120519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37a5dd-6c03-4461-9b2a-34d4f66001bd admin admin] PATH_INFO: `/v3/projects/e4369ab71f6749a3af3bf7c1e356d95c/users/4340c8004fb043e6aa21faab21c3d020/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:31.152813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37a5dd-6c03-4461-9b2a-34d4f66001bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4369ab71f6749a3af3bf7c1e356d95c, user_id=4340c8004fb043e6aa21faab21c3d020, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:31.155826 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37a5dd-6c03-4461-9b2a-34d4f66001bd admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:31.158165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:31.300615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1145/2287] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:31 2022] PUT /identity/v3/projects/e4369ab71f6749a3af3bf7c1e356d95c/users/4340c8004fb043e6aa21faab21c3d020/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:31.323937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3cc88-3630-406a-9890-731928b8b858 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:31.324322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3cc88-3630-406a-9890-731928b8b858 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:31.324525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3cc88-3630-406a-9890-731928b8b858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:31.373407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-8cb3cc88-3630-406a-9890-731928b8b858 None None] MFA Rules not processed for user `4340c8004fb043e6aa21faab21c3d020`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:31.375132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8cb3cc88-3630-406a-9890-731928b8b858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:31.395275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:31.480596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1143/2288] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:31.483683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37a5dd-6c03-4461-9b2a-34d4f66001bd admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:31.561979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37a5dd-6c03-4461-9b2a-34d4f66001bd admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:31.570121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523dc8d-66ee-4cd1-a2a6-2f919f1f2ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:31.570756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523dc8d-66ee-4cd1-a2a6-2f919f1f2ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:31.571033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523dc8d-66ee-4cd1-a2a6-2f919f1f2ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:31.571305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523dc8d-66ee-4cd1-a2a6-2f919f1f2ab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:31.571943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523dc8d-66ee-4cd1-a2a6-2f919f1f2ab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:31.573970 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523dc8d-66ee-4cd1-a2a6-2f919f1f2ab9 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:31.752954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1146/2289] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:31 2022] POST /identity/v3/projects => generated 367 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:31.754957 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3cc88-3630-406a-9890-731928b8b858 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:31.766744 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3cc88-3630-406a-9890-731928b8b858 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:31.846889 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633f0f5-4288-41cb-af21-e53a86fd1bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:31.847522 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f0f5-4288-41cb-af21-e53a86fd1bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:31.847811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f0f5-4288-41cb-af21-e53a86fd1bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:31.848092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f0f5-4288-41cb-af21-e53a86fd1bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:31.848771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633f0f5-4288-41cb-af21-e53a86fd1bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:31.851019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633f0f5-4288-41cb-af21-e53a86fd1bb4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:32.028053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1144/2290] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:31 2022] POST /identity/v3/users => generated 290 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:32.028705 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523dc8d-66ee-4cd1-a2a6-2f919f1f2ab9 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:32.034971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523dc8d-66ee-4cd1-a2a6-2f919f1f2ab9 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:32.040677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889c841-9a63-46b3-9d24-b4bff8065fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:32.041676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889c841-9a63-46b3-9d24-b4bff8065fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:32.041676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889c841-9a63-46b3-9d24-b4bff8065fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:32.041961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889c841-9a63-46b3-9d24-b4bff8065fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:32.042721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889c841-9a63-46b3-9d24-b4bff8065fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:32.044356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889c841-9a63-46b3-9d24-b4bff8065fd1 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:32.058884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1147/2291] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:32.063284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633f0f5-4288-41cb-af21-e53a86fd1bb4 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:32.075209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633f0f5-4288-41cb-af21-e53a86fd1bb4 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:32.080053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c34d71-9e8e-4487-bb61-e9f251b3ce4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:32.080705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c34d71-9e8e-4487-bb61-e9f251b3ce4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:32.081010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c34d71-9e8e-4487-bb61-e9f251b3ce4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:32.081285 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c34d71-9e8e-4487-bb61-e9f251b3ce4a admin admin] PATH_INFO: `/v3/projects/dbc097070d64447baae519473732222d/users/ca115de7617f4877a5aa3f78124b68ba/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:32.101066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c34d71-9e8e-4487-bb61-e9f251b3ce4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc097070d64447baae519473732222d, user_id=ca115de7617f4877a5aa3f78124b68ba, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:32.103796 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c34d71-9e8e-4487-bb61-e9f251b3ce4a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:32.310338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889c841-9a63-46b3-9d24-b4bff8065fd1 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:32.311940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1145/2292] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:32 2022] PUT /identity/v3/projects/dbc097070d64447baae519473732222d/users/ca115de7617f4877a5aa3f78124b68ba/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:32.331885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889c841-9a63-46b3-9d24-b4bff8065fd1 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:32.420766 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c88bab-72d5-4fe1-9924-515e204240b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:32.421500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c88bab-72d5-4fe1-9924-515e204240b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:32.421885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c88bab-72d5-4fe1-9924-515e204240b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:32.422225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c88bab-72d5-4fe1-9924-515e204240b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:32.423051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c88bab-72d5-4fe1-9924-515e204240b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:32.426628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c88bab-72d5-4fe1-9924-515e204240b8 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:32.446415 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c34d71-9e8e-4487-bb61-e9f251b3ce4a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:32.446646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1148/2293] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:32.466724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c34d71-9e8e-4487-bb61-e9f251b3ce4a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:32.472712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cfc189-d9cf-480d-9a44-cfa34518ece8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:32.473641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfc189-d9cf-480d-9a44-cfa34518ece8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:32.473881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfc189-d9cf-480d-9a44-cfa34518ece8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:32.474164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfc189-d9cf-480d-9a44-cfa34518ece8 admin admin] PATH_INFO: `/v3/projects/dbc097070d64447baae519473732222d/users/ca115de7617f4877a5aa3f78124b68ba/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:32.478858 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cfc189-d9cf-480d-9a44-cfa34518ece8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc097070d64447baae519473732222d, user_id=ca115de7617f4877a5aa3f78124b68ba, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:32.482413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cfc189-d9cf-480d-9a44-cfa34518ece8 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:32.505102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1146/2294] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:32 2022] PUT /identity/v3/projects/dbc097070d64447baae519473732222d/users/ca115de7617f4877a5aa3f78124b68ba/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:32.506933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13174dc8-e9d2-46e2-95ab-f35409dfee2a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:32.507644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13174dc8-e9d2-46e2-95ab-f35409dfee2a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:32.508127 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13174dc8-e9d2-46e2-95ab-f35409dfee2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:32.541985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-13174dc8-e9d2-46e2-95ab-f35409dfee2a None None] MFA Rules not processed for user `ca115de7617f4877a5aa3f78124b68ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:32.543434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-13174dc8-e9d2-46e2-95ab-f35409dfee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:32.605417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:32.636232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1149/2295] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:32.644986 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cfc189-d9cf-480d-9a44-cfa34518ece8 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:32.650133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b6cfc189-d9cf-480d-9a44-cfa34518ece8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:32.764504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cfc189-d9cf-480d-9a44-cfa34518ece8 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:32.769232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66851aef-7073-4c7d-93be-6ccd120b6d40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:32.769858 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66851aef-7073-4c7d-93be-6ccd120b6d40 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:32.770144 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66851aef-7073-4c7d-93be-6ccd120b6d40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:32.770418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66851aef-7073-4c7d-93be-6ccd120b6d40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:32.791589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66851aef-7073-4c7d-93be-6ccd120b6d40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:32.793083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66851aef-7073-4c7d-93be-6ccd120b6d40 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:32.811741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1147/2296] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:23:33.048330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13174dc8-e9d2-46e2-95ab-f35409dfee2a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:33.049877 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-13174dc8-e9d2-46e2-95ab-f35409dfee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:33.086208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13174dc8-e9d2-46e2-95ab-f35409dfee2a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:33.095633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b41d5-b736-4d2a-b646-3545340ba828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:33.096270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b41d5-b736-4d2a-b646-3545340ba828 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:33.096560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b41d5-b736-4d2a-b646-3545340ba828 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:33.096839 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b41d5-b736-4d2a-b646-3545340ba828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:33.169897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b41d5-b736-4d2a-b646-3545340ba828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:33.171395 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b41d5-b736-4d2a-b646-3545340ba828 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:33.183662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1150/2297] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:34.743390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66851aef-7073-4c7d-93be-6ccd120b6d40 service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:34.745335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-66851aef-7073-4c7d-93be-6ccd120b6d40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:34.762839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:34.822680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66851aef-7073-4c7d-93be-6ccd120b6d40 service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:34.827850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49da21-c7b3-43aa-b7f7-0c9c7815425b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:34.828577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49da21-c7b3-43aa-b7f7-0c9c7815425b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:34.828964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49da21-c7b3-43aa-b7f7-0c9c7815425b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:34.829258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49da21-c7b3-43aa-b7f7-0c9c7815425b admin admin] PATH_INFO: `/v3/users/71c5ac35123e425e9753f372f9c6bac0` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:34.830413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49da21-c7b3-43aa-b7f7-0c9c7815425b admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c5ac35123e425e9753f372f9c6bac0)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:34.832327 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49da21-c7b3-43aa-b7f7-0c9c7815425b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:35.180137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6f49da21-c7b3-43aa-b7f7-0c9c7815425b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c5ac35123e425e9753f372f9c6bac0'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:35.180794 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6f49da21-c7b3-43aa-b7f7-0c9c7815425b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c5ac35123e425e9753f372f9c6bac0'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:35.187617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6f49da21-c7b3-43aa-b7f7-0c9c7815425b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c5ac35123e425e9753f372f9c6bac0'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:35.188243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6f49da21-c7b3-43aa-b7f7-0c9c7815425b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c5ac35123e425e9753f372f9c6bac0'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:35.330198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6f49da21-c7b3-43aa-b7f7-0c9c7815425b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c5ac35123e425e9753f372f9c6bac0'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:35.348935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1148/2298] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:34 2022] DELETE /identity/v3/users/71c5ac35123e425e9753f372f9c6bac0 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:35.686640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b41d5-b736-4d2a-b646-3545340ba828 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:35.701246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b41d5-b736-4d2a-b646-3545340ba828 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:35.763836 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:35.764710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:35.765030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:35.765305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] PATH_INFO: `/v3/projects/297fa582000543acbf078376d7010abb` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:35.766459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] RBAC: Authorizing `identity:delete_project(project_id=297fa582000543acbf078376d7010abb)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:35.768381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:35.889089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297fa582000543acbf078376d7010abb', 'request_id': 'req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:35.894844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297fa582000543acbf078376d7010abb', 'request_id': 'req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:36.028578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297fa582000543acbf078376d7010abb', 'request_id': 'req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:36.043288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] The token cache is being invalidate because project 297fa582000543acbf078376d7010abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:36.043649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:36.044221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-07c5d5d8-4a81-4d24-921b-b4cbdd24a01e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:36.049701 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1151/2299] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:35 2022] DELETE /identity/v3/projects/297fa582000543acbf078376d7010abb => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:36.051468 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8768666-9692-4e8d-b2d5-87452f10256c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:36.051948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8768666-9692-4e8d-b2d5-87452f10256c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:36.052196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8768666-9692-4e8d-b2d5-87452f10256c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:36.072770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c8768666-9692-4e8d-b2d5-87452f10256c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:36.074074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c8768666-9692-4e8d-b2d5-87452f10256c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:36.143127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1149/2300] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:36.144824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e83217-d8c7-4267-a8cb-170fa338da84 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:36.145135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e83217-d8c7-4267-a8cb-170fa338da84 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:36.145413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e83217-d8c7-4267-a8cb-170fa338da84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:36.166965 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e5e83217-d8c7-4267-a8cb-170fa338da84 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:36.168190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e5e83217-d8c7-4267-a8cb-170fa338da84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:36.176175 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1152/2301] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:36.181326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8768666-9692-4e8d-b2d5-87452f10256c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:36.191258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8768666-9692-4e8d-b2d5-87452f10256c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:36.196940 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8adc61-06b7-4dd3-99c4-207859f40ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:36.197550 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8adc61-06b7-4dd3-99c4-207859f40ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:36.197828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8adc61-06b7-4dd3-99c4-207859f40ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:36.198097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8adc61-06b7-4dd3-99c4-207859f40ea2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:36.198746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8adc61-06b7-4dd3-99c4-207859f40ea2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:23:36.199050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8adc61-06b7-4dd3-99c4-207859f40ea2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:36.200704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8adc61-06b7-4dd3-99c4-207859f40ea2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:36.214705 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1150/2302] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:23:36.219543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e83217-d8c7-4267-a8cb-170fa338da84 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:36.229113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e83217-d8c7-4267-a8cb-170fa338da84 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:36.233848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fe150-0813-4980-ad2a-5421470d0a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:36.234446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fe150-0813-4980-ad2a-5421470d0a17 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:36.238262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fe150-0813-4980-ad2a-5421470d0a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:36.238262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fe150-0813-4980-ad2a-5421470d0a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:36.238262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fe150-0813-4980-ad2a-5421470d0a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:36.238262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fe150-0813-4980-ad2a-5421470d0a17 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:36.248261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:36.374616 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1153/2303] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:36 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:36.375822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8adc61-06b7-4dd3-99c4-207859f40ea2 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:36.386853 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8adc61-06b7-4dd3-99c4-207859f40ea2 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:36.401143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:36.447072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28e3fe-f6a0-45b8-bdce-fcd54e338a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:36.447608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28e3fe-f6a0-45b8-bdce-fcd54e338a04 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:36.447892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28e3fe-f6a0-45b8-bdce-fcd54e338a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:36.448173 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28e3fe-f6a0-45b8-bdce-fcd54e338a04 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:36.448817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28e3fe-f6a0-45b8-bdce-fcd54e338a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:36.450813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28e3fe-f6a0-45b8-bdce-fcd54e338a04 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:36.587933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1151/2304] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:36 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:36.593367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fe150-0813-4980-ad2a-5421470d0a17 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:36.603268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fe150-0813-4980-ad2a-5421470d0a17 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:36.607912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17a6fa-095e-4133-b33c-9fd2d82ce80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:36.608508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17a6fa-095e-4133-b33c-9fd2d82ce80a admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:36.608787 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17a6fa-095e-4133-b33c-9fd2d82ce80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:36.609088 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17a6fa-095e-4133-b33c-9fd2d82ce80a admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:36.609831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17a6fa-095e-4133-b33c-9fd2d82ce80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:36.611688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17a6fa-095e-4133-b33c-9fd2d82ce80a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:36.622230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1154/2305] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:36.626456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28e3fe-f6a0-45b8-bdce-fcd54e338a04 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:36.636488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28e3fe-f6a0-45b8-bdce-fcd54e338a04 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:36.641559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bafea-5267-4e3e-bf33-1f7c95beb32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:36.642159 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bafea-5267-4e3e-bf33-1f7c95beb32c admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:36.650438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bafea-5267-4e3e-bf33-1f7c95beb32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:36.650438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bafea-5267-4e3e-bf33-1f7c95beb32c admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:36.650438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bafea-5267-4e3e-bf33-1f7c95beb32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:36.650438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bafea-5267-4e3e-bf33-1f7c95beb32c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:36.659377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1152/2306] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:36.664120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17a6fa-095e-4133-b33c-9fd2d82ce80a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:36.674073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17a6fa-095e-4133-b33c-9fd2d82ce80a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:36.679249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e3084-462c-4ea3-87f9-b99fa710dc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:36.679884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e3084-462c-4ea3-87f9-b99fa710dc8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:36.680163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e3084-462c-4ea3-87f9-b99fa710dc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:36.680432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e3084-462c-4ea3-87f9-b99fa710dc8a admin admin] PATH_INFO: `/v3/projects/8acb53c7ce5e4866a5bef0e240216341/users/ddfe768a4b7643ce89d98ebfef9760a4/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:36.695144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e3084-462c-4ea3-87f9-b99fa710dc8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8acb53c7ce5e4866a5bef0e240216341, user_id=ddfe768a4b7643ce89d98ebfef9760a4, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:36.698079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e3084-462c-4ea3-87f9-b99fa710dc8a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:36.862739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1155/2307] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:36 2022] PUT /identity/v3/projects/8acb53c7ce5e4866a5bef0e240216341/users/ddfe768a4b7643ce89d98ebfef9760a4/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:36.864478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491b302-7e69-4277-93e2-643e65dac753 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:36.864764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491b302-7e69-4277-93e2-643e65dac753 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:36.865069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491b302-7e69-4277-93e2-643e65dac753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:36.892510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a491b302-7e69-4277-93e2-643e65dac753 None None] MFA Rules not processed for user `ddfe768a4b7643ce89d98ebfef9760a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:36.893819 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a491b302-7e69-4277-93e2-643e65dac753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:36.960055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1153/2308] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:36.971107 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e3084-462c-4ea3-87f9-b99fa710dc8a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:36.972985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e22e3084-462c-4ea3-87f9-b99fa710dc8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:37.058271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e3084-462c-4ea3-87f9-b99fa710dc8a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:37.064791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925f984-2a63-4202-9d97-e66d91490488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:37.065432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925f984-2a63-4202-9d97-e66d91490488 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:37.065713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925f984-2a63-4202-9d97-e66d91490488 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:37.066019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925f984-2a63-4202-9d97-e66d91490488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:37.075245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925f984-2a63-4202-9d97-e66d91490488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:37.076830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925f984-2a63-4202-9d97-e66d91490488 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:37.087182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1156/2309] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:37.338642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491b302-7e69-4277-93e2-643e65dac753 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:37.403484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491b302-7e69-4277-93e2-643e65dac753 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:37.408235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:37.408893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:37.409171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:37.409444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] PATH_INFO: `/v3/users/ddfe768a4b7643ce89d98ebfef9760a4` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:37.410697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddfe768a4b7643ce89d98ebfef9760a4)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:37.412826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:37.539951 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:37.640375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddfe768a4b7643ce89d98ebfef9760a4'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:37.641048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddfe768a4b7643ce89d98ebfef9760a4'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:37.648479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddfe768a4b7643ce89d98ebfef9760a4'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:37.649198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddfe768a4b7643ce89d98ebfef9760a4'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:37.729049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddfe768a4b7643ce89d98ebfef9760a4'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:37.745471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1154/2310] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:37 2022] DELETE /identity/v3/users/ddfe768a4b7643ce89d98ebfef9760a4 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:37.749962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853668f-60d1-4b62-a0c6-9ed79ea12cca None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:37.750415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853668f-60d1-4b62-a0c6-9ed79ea12cca None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:37.750631 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853668f-60d1-4b62-a0c6-9ed79ea12cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:37.770184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0853668f-60d1-4b62-a0c6-9ed79ea12cca None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:37.771397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0853668f-60d1-4b62-a0c6-9ed79ea12cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:37.829717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1157/2311] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:37.843134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:37.848568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:37.918096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa7227-e429-4b9c-b63b-40a978c0cc7c admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:37.921908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73f6d8-fbee-4e79-ab81-71006e658ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:37.922510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f6d8-fbee-4e79-ab81-71006e658ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:37.922792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f6d8-fbee-4e79-ab81-71006e658ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:37.923066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f6d8-fbee-4e79-ab81-71006e658ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:37.932616 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73f6d8-fbee-4e79-ab81-71006e658ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:37.934084 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73f6d8-fbee-4e79-ab81-71006e658ca7 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:37.943534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1155/2312] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:38.586358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853668f-60d1-4b62-a0c6-9ed79ea12cca None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:38.588261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0853668f-60d1-4b62-a0c6-9ed79ea12cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:38.607896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853668f-60d1-4b62-a0c6-9ed79ea12cca None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:38.614190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:38.614920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:38.615286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:38.615734 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] PATH_INFO: `/v3/projects/8acb53c7ce5e4866a5bef0e240216341` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:38.617421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8acb53c7ce5e4866a5bef0e240216341)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:38.620310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:38.779668 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:38.791975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8acb53c7ce5e4866a5bef0e240216341', 'request_id': 'req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:38.797637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8acb53c7ce5e4866a5bef0e240216341', 'request_id': 'req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:38.906488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8acb53c7ce5e4866a5bef0e240216341', 'request_id': 'req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:38.917730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] The token cache is being invalidate because project 8acb53c7ce5e4866a5bef0e240216341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:38.918096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:38.918693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-70788cd4-d1e6-4e3f-a4d6-6993da92baa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:38.920339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1158/2313] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:38 2022] DELETE /identity/v3/projects/8acb53c7ce5e4866a5bef0e240216341 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:38.925518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16997424-dd74-44e9-be60-452eeace13d6 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:38.925993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16997424-dd74-44e9-be60-452eeace13d6 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:38.926105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16997424-dd74-44e9-be60-452eeace13d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:38.948779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-16997424-dd74-44e9-be60-452eeace13d6 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:38.950037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-16997424-dd74-44e9-be60-452eeace13d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:39.015348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1156/2314] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:39.024549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0abb1-d49a-4872-8bcf-760692b0c8ed None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.025000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0abb1-d49a-4872-8bcf-760692b0c8ed None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.025372 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0abb1-d49a-4872-8bcf-760692b0c8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.046570 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e4b0abb1-d49a-4872-8bcf-760692b0c8ed None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:39.047848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e4b0abb1-d49a-4872-8bcf-760692b0c8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:39.056718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1159/2315] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:39.061116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16997424-dd74-44e9-be60-452eeace13d6 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:39.071358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16997424-dd74-44e9-be60-452eeace13d6 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:39.076650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a79bcb-2f28-4c6f-88dd-57ff5353ff7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:39.077308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a79bcb-2f28-4c6f-88dd-57ff5353ff7b admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.077590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a79bcb-2f28-4c6f-88dd-57ff5353ff7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.077866 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a79bcb-2f28-4c6f-88dd-57ff5353ff7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.087903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a79bcb-2f28-4c6f-88dd-57ff5353ff7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:23:39.087903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a79bcb-2f28-4c6f-88dd-57ff5353ff7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:39.087903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a79bcb-2f28-4c6f-88dd-57ff5353ff7b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:39.105185 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1157/2316] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:23:39.105401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0abb1-d49a-4872-8bcf-760692b0c8ed None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:39.110694 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0abb1-d49a-4872-8bcf-760692b0c8ed None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:39.116459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65039ddf-d36b-4b0f-b46c-f1531338578c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:39.125128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65039ddf-d36b-4b0f-b46c-f1531338578c admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.125128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65039ddf-d36b-4b0f-b46c-f1531338578c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.125128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65039ddf-d36b-4b0f-b46c-f1531338578c admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.125128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65039ddf-d36b-4b0f-b46c-f1531338578c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:39.125128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65039ddf-d36b-4b0f-b46c-f1531338578c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:39.242051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1160/2317] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:39 2022] POST /identity/v3/projects => generated 403 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:39.246328 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a79bcb-2f28-4c6f-88dd-57ff5353ff7b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:39.256993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a79bcb-2f28-4c6f-88dd-57ff5353ff7b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:39.292933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65039ddf-d36b-4b0f-b46c-f1531338578c admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:39.294850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-65039ddf-d36b-4b0f-b46c-f1531338578c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:39.322543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:39.348540 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91aa19-0f4f-46fe-92f1-0d013fb0acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:39.349444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91aa19-0f4f-46fe-92f1-0d013fb0acc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.350516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91aa19-0f4f-46fe-92f1-0d013fb0acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.350785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91aa19-0f4f-46fe-92f1-0d013fb0acc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.351425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91aa19-0f4f-46fe-92f1-0d013fb0acc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:39.353352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91aa19-0f4f-46fe-92f1-0d013fb0acc1 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:39.375072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65039ddf-d36b-4b0f-b46c-f1531338578c admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:39.379855 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b97e20-6205-4e2c-8943-8f3bde66bb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:39.380490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97e20-6205-4e2c-8943-8f3bde66bb80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.380772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97e20-6205-4e2c-8943-8f3bde66bb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.381230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97e20-6205-4e2c-8943-8f3bde66bb80 admin admin] PATH_INFO: `/v3/users/4340c8004fb043e6aa21faab21c3d020` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.382450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b97e20-6205-4e2c-8943-8f3bde66bb80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4340c8004fb043e6aa21faab21c3d020)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:39.384584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b97e20-6205-4e2c-8943-8f3bde66bb80 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:39.511253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1158/2318] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:39 2022] POST /identity/v3/users => generated 302 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:39.512694 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91aa19-0f4f-46fe-92f1-0d013fb0acc1 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:39.522786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91aa19-0f4f-46fe-92f1-0d013fb0acc1 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:39.527613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda80370-76c3-4f4c-909f-bd5e17603132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:39.528441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda80370-76c3-4f4c-909f-bd5e17603132 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.528731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda80370-76c3-4f4c-909f-bd5e17603132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.529047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda80370-76c3-4f4c-909f-bd5e17603132 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.529997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda80370-76c3-4f4c-909f-bd5e17603132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:39.531917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda80370-76c3-4f4c-909f-bd5e17603132 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:39.544649 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1159/2319] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:39.551198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda80370-76c3-4f4c-909f-bd5e17603132 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:39.562511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda80370-76c3-4f4c-909f-bd5e17603132 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:39.568093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec3943-8763-4a91-ae15-61438f7965b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:39.568093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec3943-8763-4a91-ae15-61438f7965b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.568465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec3943-8763-4a91-ae15-61438f7965b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.568465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec3943-8763-4a91-ae15-61438f7965b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.569514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec3943-8763-4a91-ae15-61438f7965b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:39.571111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec3943-8763-4a91-ae15-61438f7965b6 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:39.581673 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1160/2320] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:39.586172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec3943-8763-4a91-ae15-61438f7965b6 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:39.596036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec3943-8763-4a91-ae15-61438f7965b6 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:39.602000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a2c22-2f8c-4fc2-99e2-bf3182532406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:39.602679 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a2c22-2f8c-4fc2-99e2-bf3182532406 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.602960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a2c22-2f8c-4fc2-99e2-bf3182532406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.603237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a2c22-2f8c-4fc2-99e2-bf3182532406 admin admin] PATH_INFO: `/v3/projects/12e1b900f66c45daafbc0e540abb115e/users/64a0f08fde5e4a409900aea874a82b67/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.619591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a2c22-2f8c-4fc2-99e2-bf3182532406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e1b900f66c45daafbc0e540abb115e, user_id=64a0f08fde5e4a409900aea874a82b67, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:39.622343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a2c22-2f8c-4fc2-99e2-bf3182532406 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:39.666105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-16b97e20-6205-4e2c-8943-8f3bde66bb80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4340c8004fb043e6aa21faab21c3d020'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:39.666773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-16b97e20-6205-4e2c-8943-8f3bde66bb80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4340c8004fb043e6aa21faab21c3d020'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:39.673918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-16b97e20-6205-4e2c-8943-8f3bde66bb80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4340c8004fb043e6aa21faab21c3d020'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:39.688071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-16b97e20-6205-4e2c-8943-8f3bde66bb80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4340c8004fb043e6aa21faab21c3d020'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:39.755007 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1161/2321] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:39 2022] PUT /identity/v3/projects/12e1b900f66c45daafbc0e540abb115e/users/64a0f08fde5e4a409900aea874a82b67/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:39.767847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3421171-d916-4afc-86c8-cf965eaacb27 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.768072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3421171-d916-4afc-86c8-cf965eaacb27 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.768352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3421171-d916-4afc-86c8-cf965eaacb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.788561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-16b97e20-6205-4e2c-8943-8f3bde66bb80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4340c8004fb043e6aa21faab21c3d020'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:39.795875 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1161/2322] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:39 2022] DELETE /identity/v3/users/4340c8004fb043e6aa21faab21c3d020 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:39.798497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a3421171-d916-4afc-86c8-cf965eaacb27 None None] MFA Rules not processed for user `64a0f08fde5e4a409900aea874a82b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:39.799699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a3421171-d916-4afc-86c8-cf965eaacb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:39.800985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e109775-0db9-45e7-b905-4262eda8f390 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.801450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e109775-0db9-45e7-b905-4262eda8f390 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.801832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e109775-0db9-45e7-b905-4262eda8f390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.821523 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3e109775-0db9-45e7-b905-4262eda8f390 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:39.822727 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3e109775-0db9-45e7-b905-4262eda8f390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:39.865398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1162/2323] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:39 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:39.875395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3421171-d916-4afc-86c8-cf965eaacb27 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:39.875395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a3421171-d916-4afc-86c8-cf965eaacb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:39.883091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1162/2324] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:39.892722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e109775-0db9-45e7-b905-4262eda8f390 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:39.894259 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3e109775-0db9-45e7-b905-4262eda8f390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:39.952039 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3421171-d916-4afc-86c8-cf965eaacb27 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:39.953821 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e109775-0db9-45e7-b905-4262eda8f390 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:39.956850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b769f6c-2f4e-48ed-8676-141c24d37dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:39.957729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41bc89-e276-4213-8e43-0bb44fa38cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:39.957892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b769f6c-2f4e-48ed-8676-141c24d37dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.957987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b769f6c-2f4e-48ed-8676-141c24d37dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.958239 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41bc89-e276-4213-8e43-0bb44fa38cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.958380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b769f6c-2f4e-48ed-8676-141c24d37dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.958513 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41bc89-e276-4213-8e43-0bb44fa38cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.958789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41bc89-e276-4213-8e43-0bb44fa38cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.958908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b769f6c-2f4e-48ed-8676-141c24d37dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:39.961877 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b769f6c-2f4e-48ed-8676-141c24d37dc1 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:39.968662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41bc89-e276-4213-8e43-0bb44fa38cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:39.970134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41bc89-e276-4213-8e43-0bb44fa38cd7 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:39.980478 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1163/2325] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:23:40.093819 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1163/2326] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:39 2022] POST /identity/v3/projects => generated 405 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:40.096327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41bc89-e276-4213-8e43-0bb44fa38cd7 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:40.106483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41bc89-e276-4213-8e43-0bb44fa38cd7 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:40.164851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb682d87-653d-43dc-a4fe-af176d515a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:40.165492 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb682d87-653d-43dc-a4fe-af176d515a80 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:40.165774 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb682d87-653d-43dc-a4fe-af176d515a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:40.166051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb682d87-653d-43dc-a4fe-af176d515a80 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:40.166715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb682d87-653d-43dc-a4fe-af176d515a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:40.169003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb682d87-653d-43dc-a4fe-af176d515a80 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:40.196341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:40.311160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1164/2327] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:40 2022] POST /identity/v3/users => generated 309 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:40.316072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b769f6c-2f4e-48ed-8676-141c24d37dc1 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:40.329379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b769f6c-2f4e-48ed-8676-141c24d37dc1 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:40.335909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9773e1-49fd-494d-970e-af682d2dbdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:40.336562 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9773e1-49fd-494d-970e-af682d2dbdd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:40.340585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9773e1-49fd-494d-970e-af682d2dbdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:40.340585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9773e1-49fd-494d-970e-af682d2dbdd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:40.340585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9773e1-49fd-494d-970e-af682d2dbdd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:40.340585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9773e1-49fd-494d-970e-af682d2dbdd0 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:40.351751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1164/2328] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:40.356535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb682d87-653d-43dc-a4fe-af176d515a80 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:40.367733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb682d87-653d-43dc-a4fe-af176d515a80 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:40.372549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403f4df-7509-40a2-8edd-e833f4b22c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:40.373250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403f4df-7509-40a2-8edd-e833f4b22c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:40.373535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403f4df-7509-40a2-8edd-e833f4b22c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:40.373813 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403f4df-7509-40a2-8edd-e833f4b22c68 admin admin] PATH_INFO: `/v3/projects/4e6e0335fc71495f8904ea5ebd0879df/users/8049257ec5954f2fb2150f3471baa7ac/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:40.389877 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403f4df-7509-40a2-8edd-e833f4b22c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6e0335fc71495f8904ea5ebd0879df, user_id=8049257ec5954f2fb2150f3471baa7ac, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:40.392570 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403f4df-7509-40a2-8edd-e833f4b22c68 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:40.429674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9773e1-49fd-494d-970e-af682d2dbdd0 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:40.431372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-eb9773e1-49fd-494d-970e-af682d2dbdd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:40.444002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9773e1-49fd-494d-970e-af682d2dbdd0 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:40.448730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:40.449401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:40.449809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:40.450070 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] PATH_INFO: `/v3/projects/e4369ab71f6749a3af3bf7c1e356d95c` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:40.451176 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4369ab71f6749a3af3bf7c1e356d95c)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:40.453100 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:40.457116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1165/2329] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:40 2022] PUT /identity/v3/projects/4e6e0335fc71495f8904ea5ebd0879df/users/8049257ec5954f2fb2150f3471baa7ac/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:40.461566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403f4df-7509-40a2-8edd-e833f4b22c68 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:40.528994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403f4df-7509-40a2-8edd-e833f4b22c68 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:40.533929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6c01d-6f08-498a-92a5-ce724e9e2699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:40.535328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6c01d-6f08-498a-92a5-ce724e9e2699 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:40.535667 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6c01d-6f08-498a-92a5-ce724e9e2699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:40.535953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6c01d-6f08-498a-92a5-ce724e9e2699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:40.536602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6c01d-6f08-498a-92a5-ce724e9e2699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:40.538905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6c01d-6f08-498a-92a5-ce724e9e2699 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:40.549181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1166/2330] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:40.554564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6c01d-6f08-498a-92a5-ce724e9e2699 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:40.564081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6c01d-6f08-498a-92a5-ce724e9e2699 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:40.570990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa9100-6faa-4966-81b1-edec17ce546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:40.570990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa9100-6faa-4966-81b1-edec17ce546f admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:40.570990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa9100-6faa-4966-81b1-edec17ce546f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:40.570990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa9100-6faa-4966-81b1-edec17ce546f admin admin] PATH_INFO: `/v3/projects/4e6e0335fc71495f8904ea5ebd0879df/users/8049257ec5954f2fb2150f3471baa7ac/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:40.572213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa9100-6faa-4966-81b1-edec17ce546f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6e0335fc71495f8904ea5ebd0879df, user_id=8049257ec5954f2fb2150f3471baa7ac, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:40.574995 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa9100-6faa-4966-81b1-edec17ce546f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:40.584997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1167/2331] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:40 2022] PUT /identity/v3/projects/4e6e0335fc71495f8904ea5ebd0879df/users/8049257ec5954f2fb2150f3471baa7ac/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:40.595473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eee05c-ed50-42c9-ae40-d2ebcc27f29f None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:40.598150 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eee05c-ed50-42c9-ae40-d2ebcc27f29f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:40.598150 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eee05c-ed50-42c9-ae40-d2ebcc27f29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:40.607911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4369ab71f6749a3af3bf7c1e356d95c', 'request_id': 'req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:40.613811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4369ab71f6749a3af3bf7c1e356d95c', 'request_id': 'req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:40.621386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4369ab71f6749a3af3bf7c1e356d95c', 'request_id': 'req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:40.626604 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-87eee05c-ed50-42c9-ae40-d2ebcc27f29f None None] MFA Rules not processed for user `8049257ec5954f2fb2150f3471baa7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:40.627865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-87eee05c-ed50-42c9-ae40-d2ebcc27f29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:40.697060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1168/2332] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:40 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:40.708151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eee05c-ed50-42c9-ae40-d2ebcc27f29f None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:40.713977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-87eee05c-ed50-42c9-ae40-d2ebcc27f29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:40.746059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] The token cache is being invalidate because project e4369ab71f6749a3af3bf7c1e356d95c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:40.746323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:40.746999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:40.750943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1165/2333] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:40 2022] DELETE /identity/v3/projects/e4369ab71f6749a3af3bf7c1e356d95c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:40.753944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:40.755381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:40.776254 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eee05c-ed50-42c9-ae40-d2ebcc27f29f None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:40.780912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa7a27-d0bf-487e-8c6f-1fdcdac2050c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:40.781739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa7a27-d0bf-487e-8c6f-1fdcdac2050c service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:40.781946 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa7a27-d0bf-487e-8c6f-1fdcdac2050c service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:40.782233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa7a27-d0bf-487e-8c6f-1fdcdac2050c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:40.784632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-27aa7a27-d0bf-487e-8c6f-1fdcdac2050c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:40.808927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa7a27-d0bf-487e-8c6f-1fdcdac2050c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:40.811439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa7a27-d0bf-487e-8c6f-1fdcdac2050c service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:40.827253 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1169/2334] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:40 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:23:40.887847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:40.924652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc405653-f2d1-4f9a-ab2b-ce7724c27d35 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:40.935650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dfd86-f658-44e1-baa9-125f4a9225da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:40.936342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dfd86-f658-44e1-baa9-125f4a9225da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:40.936631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dfd86-f658-44e1-baa9-125f4a9225da admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:40.936949 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dfd86-f658-44e1-baa9-125f4a9225da admin admin] PATH_INFO: `/v3/users/ca115de7617f4877a5aa3f78124b68ba` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:40.938431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2dfd86-f658-44e1-baa9-125f4a9225da admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca115de7617f4877a5aa3f78124b68ba)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:40.940406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2dfd86-f658-44e1-baa9-125f4a9225da admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:41.008000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa7a27-d0bf-487e-8c6f-1fdcdac2050c service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:41.009495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-27aa7a27-d0bf-487e-8c6f-1fdcdac2050c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:41.023443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa7a27-d0bf-487e-8c6f-1fdcdac2050c service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:41.028134 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:41.028778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:41.029095 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:41.029370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] PATH_INFO: `/v3/users/64a0f08fde5e4a409900aea874a82b67` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:41.030728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a0f08fde5e4a409900aea874a82b67)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:41.032916 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:41.328571 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0f2dfd86-f658-44e1-baa9-125f4a9225da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca115de7617f4877a5aa3f78124b68ba'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:41.329361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0f2dfd86-f658-44e1-baa9-125f4a9225da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca115de7617f4877a5aa3f78124b68ba'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:41.341880 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0f2dfd86-f658-44e1-baa9-125f4a9225da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca115de7617f4877a5aa3f78124b68ba'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:41.342607 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0f2dfd86-f658-44e1-baa9-125f4a9225da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca115de7617f4877a5aa3f78124b68ba'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:41.355084 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a0f08fde5e4a409900aea874a82b67'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:41.356456 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a0f08fde5e4a409900aea874a82b67'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:41.368494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a0f08fde5e4a409900aea874a82b67'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:41.383814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a0f08fde5e4a409900aea874a82b67'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:41.440570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0f2dfd86-f658-44e1-baa9-125f4a9225da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca115de7617f4877a5aa3f78124b68ba'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:41.465816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1166/2335] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:40 2022] DELETE /identity/v3/users/ca115de7617f4877a5aa3f78124b68ba => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:41.588952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a0f08fde5e4a409900aea874a82b67'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:41.590766 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:41.597176 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1170/2336] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:41 2022] DELETE /identity/v3/users/64a0f08fde5e4a409900aea874a82b67 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:41.602149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79306d92-08b9-4f79-b36d-b124189fcb66 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:41.602438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79306d92-08b9-4f79-b36d-b124189fcb66 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:41.602711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79306d92-08b9-4f79-b36d-b124189fcb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:41.623142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-79306d92-08b9-4f79-b36d-b124189fcb66 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:41.624367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-79306d92-08b9-4f79-b36d-b124189fcb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:41.685488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1167/2337] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:41.696242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:41.698246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:41.823508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b6119-9672-4a14-8ef5-58458dd2f577 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:41.829972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c850db-3230-4f35-8815-082c268ddf64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:41.831846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c850db-3230-4f35-8815-082c268ddf64 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:41.832328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c850db-3230-4f35-8815-082c268ddf64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:41.832790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c850db-3230-4f35-8815-082c268ddf64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:41.847301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c850db-3230-4f35-8815-082c268ddf64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:41.848741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c850db-3230-4f35-8815-082c268ddf64 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:41.859098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1171/2338] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:23:41.953133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:INSERT=3,keystone:SELECT=82,keystone:DELETE=15 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:42.456699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:42.654859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79306d92-08b9-4f79-b36d-b124189fcb66 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:42.657087 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-79306d92-08b9-4f79-b36d-b124189fcb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:42.673109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79306d92-08b9-4f79-b36d-b124189fcb66 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:42.678641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:42.679681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:42.680141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:42.680588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] PATH_INFO: `/v3/projects/dbc097070d64447baae519473732222d` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:42.682061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbc097070d64447baae519473732222d)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:42.684486 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:42.779630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c850db-3230-4f35-8815-082c268ddf64 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:42.781327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-36c850db-3230-4f35-8815-082c268ddf64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:42.806187 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c850db-3230-4f35-8815-082c268ddf64 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:42.811909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:42.812502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:42.812891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:42.813254 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] PATH_INFO: `/v3/projects/12e1b900f66c45daafbc0e540abb115e` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:42.814504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12e1b900f66c45daafbc0e540abb115e)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:42.817358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:42.862437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbc097070d64447baae519473732222d', 'request_id': 'req-db5ddef9-7fd7-4fc9-9926-10191e4fc139'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:42.869713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbc097070d64447baae519473732222d', 'request_id': 'req-db5ddef9-7fd7-4fc9-9926-10191e4fc139'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:42.877067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbc097070d64447baae519473732222d', 'request_id': 'req-db5ddef9-7fd7-4fc9-9926-10191e4fc139'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:42.955245 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] The token cache is being invalidate because project dbc097070d64447baae519473732222d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:42.962337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:42.962337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-db5ddef9-7fd7-4fc9-9926-10191e4fc139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:42.963217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1168/2339] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:42 2022] DELETE /identity/v3/projects/dbc097070d64447baae519473732222d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:42.965558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671648a-10c8-4bcf-be93-415f9be6e60a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:42.966029 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671648a-10c8-4bcf-be93-415f9be6e60a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:42.966526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671648a-10c8-4bcf-be93-415f9be6e60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:42.972328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12e1b900f66c45daafbc0e540abb115e', 'request_id': 'req-58f363e6-199f-4a4b-86df-127fab820a45'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:42.982346 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12e1b900f66c45daafbc0e540abb115e', 'request_id': 'req-58f363e6-199f-4a4b-86df-127fab820a45'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:42.995920 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-3671648a-10c8-4bcf-be93-415f9be6e60a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:42.997458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3671648a-10c8-4bcf-be93-415f9be6e60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:43.068557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1169/2340] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:43.076826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2a0b1-812f-4302-8c26-d3aea5bcfb4c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:43.077167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2a0b1-812f-4302-8c26-d3aea5bcfb4c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:43.077524 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2a0b1-812f-4302-8c26-d3aea5bcfb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:43.099205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-cbb2a0b1-812f-4302-8c26-d3aea5bcfb4c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:43.101553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-cbb2a0b1-812f-4302-8c26-d3aea5bcfb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:43.111556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1170/2341] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:43.115297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2a0b1-812f-4302-8c26-d3aea5bcfb4c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:43.121883 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12e1b900f66c45daafbc0e540abb115e', 'request_id': 'req-58f363e6-199f-4a4b-86df-127fab820a45'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:43.127242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2a0b1-812f-4302-8c26-d3aea5bcfb4c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:43.133276 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baa4c5-0f1f-4bda-94f0-e2f1095e54da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:43.133887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa4c5-0f1f-4bda-94f0-e2f1095e54da admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:43.134161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa4c5-0f1f-4bda-94f0-e2f1095e54da admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:43.134811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa4c5-0f1f-4bda-94f0-e2f1095e54da admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:43.135077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baa4c5-0f1f-4bda-94f0-e2f1095e54da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:23:43.135379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baa4c5-0f1f-4bda-94f0-e2f1095e54da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:43.136118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] The token cache is being invalidate because project 12e1b900f66c45daafbc0e540abb115e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:43.136414 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:43.137211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baa4c5-0f1f-4bda-94f0-e2f1095e54da admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:43.137452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:43.142377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1172/2342] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:42 2022] DELETE /identity/v3/projects/12e1b900f66c45daafbc0e540abb115e => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:43.143225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:43.144580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:43.154906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1171/2343] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:23:43.156423 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baa4c5-0f1f-4bda-94f0-e2f1095e54da admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:43.158083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f363e6-199f-4a4b-86df-127fab820a45 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:43.158429 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-05baa4c5-0f1f-4bda-94f0-e2f1095e54da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:43.163927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:43.164852 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:43.165263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:43.165541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] PATH_INFO: `/v3/users/8049257ec5954f2fb2150f3471baa7ac` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:43.166980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8049257ec5954f2fb2150f3471baa7ac)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:43.168917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:43.173419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baa4c5-0f1f-4bda-94f0-e2f1095e54da admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:43.179386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac37988-fe4d-404b-8b11-f03d21d42e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:43.180484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac37988-fe4d-404b-8b11-f03d21d42e06 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:43.180484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac37988-fe4d-404b-8b11-f03d21d42e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:43.182488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac37988-fe4d-404b-8b11-f03d21d42e06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:43.182488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac37988-fe4d-404b-8b11-f03d21d42e06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:43.183327 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac37988-fe4d-404b-8b11-f03d21d42e06 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:43.294696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1172/2344] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:43 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:43.300481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac37988-fe4d-404b-8b11-f03d21d42e06 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:43.367037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac37988-fe4d-404b-8b11-f03d21d42e06 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:43.371911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40cf59-b770-4162-8194-c5db4438dc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:43.372517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40cf59-b770-4162-8194-c5db4438dc48 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:43.372801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40cf59-b770-4162-8194-c5db4438dc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:43.373106 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40cf59-b770-4162-8194-c5db4438dc48 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:43.373810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40cf59-b770-4162-8194-c5db4438dc48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:43.375841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40cf59-b770-4162-8194-c5db4438dc48 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:43.393855 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8049257ec5954f2fb2150f3471baa7ac'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:43.394632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8049257ec5954f2fb2150f3471baa7ac'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:43.402686 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8049257ec5954f2fb2150f3471baa7ac'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:43.416876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8049257ec5954f2fb2150f3471baa7ac'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:43.519919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8049257ec5954f2fb2150f3471baa7ac'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:43.530200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1173/2345] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:43 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:43.534504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1173/2346] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:43 2022] DELETE /identity/v3/users/8049257ec5954f2fb2150f3471baa7ac => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:43.536043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40cf59-b770-4162-8194-c5db4438dc48 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:43.537931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2d40cf59-b770-4162-8194-c5db4438dc48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:43.560989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40cf59-b770-4162-8194-c5db4438dc48 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:43.606558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:43.686696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c7dc5-6875-480e-8f43-8dca62c66417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:43.688186 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c7dc5-6875-480e-8f43-8dca62c66417 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:43.688948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c7dc5-6875-480e-8f43-8dca62c66417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:43.691341 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c7dc5-6875-480e-8f43-8dca62c66417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:43.692846 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c7dc5-6875-480e-8f43-8dca62c66417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:43.696213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c7dc5-6875-480e-8f43-8dca62c66417 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:43.711741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1174/2347] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:43.715732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:43.727507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26952a32-436b-4ea2-8bdf-9215d9b32e0d admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:43.732839 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85fc31-0b76-41c3-adf1-b2e7c78f590b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:43.733540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85fc31-0b76-41c3-adf1-b2e7c78f590b admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:43.733824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85fc31-0b76-41c3-adf1-b2e7c78f590b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:43.734528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85fc31-0b76-41c3-adf1-b2e7c78f590b admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:43.735190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85fc31-0b76-41c3-adf1-b2e7c78f590b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:43.737294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85fc31-0b76-41c3-adf1-b2e7c78f590b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:43.749547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1174/2348] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:43.753805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c7dc5-6875-480e-8f43-8dca62c66417 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:43.765286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c7dc5-6875-480e-8f43-8dca62c66417 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:43.770114 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59b14a-eacb-4ebf-af2e-d13f2c985adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:43.775716 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59b14a-eacb-4ebf-af2e-d13f2c985adb admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:43.775716 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59b14a-eacb-4ebf-af2e-d13f2c985adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:43.775716 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59b14a-eacb-4ebf-af2e-d13f2c985adb admin admin] PATH_INFO: `/v3/projects/44b040511d164d2b9d0061c8dc80e4ca/users/e765c4b152d340fe8bd4cca003d4c446/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:43.790935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59b14a-eacb-4ebf-af2e-d13f2c985adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b040511d164d2b9d0061c8dc80e4ca, user_id=e765c4b152d340fe8bd4cca003d4c446, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:43.793748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59b14a-eacb-4ebf-af2e-d13f2c985adb admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:43.910330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1175/2349] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:43 2022] PUT /identity/v3/projects/44b040511d164d2b9d0061c8dc80e4ca/users/e765c4b152d340fe8bd4cca003d4c446/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:43.924080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ac7e6-2ea7-4bd4-ba1e-42f64c29973b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:43.924469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ac7e6-2ea7-4bd4-ba1e-42f64c29973b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:43.924838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ac7e6-2ea7-4bd4-ba1e-42f64c29973b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:43.956700 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-476ac7e6-2ea7-4bd4-ba1e-42f64c29973b None None] MFA Rules not processed for user `e765c4b152d340fe8bd4cca003d4c446`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:43.958097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-476ac7e6-2ea7-4bd4-ba1e-42f64c29973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:44.024515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1175/2350] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:44.028960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59b14a-eacb-4ebf-af2e-d13f2c985adb admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:44.092104 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59b14a-eacb-4ebf-af2e-d13f2c985adb admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:44.097022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a83c2b-54a3-4653-a101-1bc9f6b622c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:44.097708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a83c2b-54a3-4653-a101-1bc9f6b622c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.097959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a83c2b-54a3-4653-a101-1bc9f6b622c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.098236 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a83c2b-54a3-4653-a101-1bc9f6b622c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.098877 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a83c2b-54a3-4653-a101-1bc9f6b622c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:44.101662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a83c2b-54a3-4653-a101-1bc9f6b622c6 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:44.159112 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ac7e6-2ea7-4bd4-ba1e-42f64c29973b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:44.160504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-476ac7e6-2ea7-4bd4-ba1e-42f64c29973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:44.174779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ac7e6-2ea7-4bd4-ba1e-42f64c29973b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:44.180501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:44.181179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.181459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.181735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] PATH_INFO: `/v3/projects/4e6e0335fc71495f8904ea5ebd0879df` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.182877 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6e0335fc71495f8904ea5ebd0879df)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:44.184798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:44.242514 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1176/2351] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:44 2022] POST /identity/v3/projects => generated 387 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:44.246675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a83c2b-54a3-4653-a101-1bc9f6b622c6 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:44.309143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a83c2b-54a3-4653-a101-1bc9f6b622c6 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:44.314737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e2e74-f32b-4f22-b367-9299ea67a688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:44.315452 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e2e74-f32b-4f22-b367-9299ea67a688 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.315848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e2e74-f32b-4f22-b367-9299ea67a688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.316238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e2e74-f32b-4f22-b367-9299ea67a688 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.317304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e2e74-f32b-4f22-b367-9299ea67a688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:44.321543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e2e74-f32b-4f22-b367-9299ea67a688 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:44.327916 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6e0335fc71495f8904ea5ebd0879df', 'request_id': 'req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:44.333272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6e0335fc71495f8904ea5ebd0879df', 'request_id': 'req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:44.375697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:44.404165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6e0335fc71495f8904ea5ebd0879df', 'request_id': 'req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:44.410771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1177/2352] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:44 2022] POST /identity/v3/users => generated 300 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:44.415371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e2e74-f32b-4f22-b367-9299ea67a688 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:44.421998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] The token cache is being invalidate because project 4e6e0335fc71495f8904ea5ebd0879df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:44.421998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:44.422578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-9dfd0afa-60f4-498c-ad56-ffa85b8c0545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:44.436608 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1176/2353] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:44 2022] DELETE /identity/v3/projects/4e6e0335fc71495f8904ea5ebd0879df => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:44.436989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e2e74-f32b-4f22-b367-9299ea67a688 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:44.439612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659df41-65a4-43d1-92a3-d9e839e3265f None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.439904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659df41-65a4-43d1-92a3-d9e839e3265f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.440186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659df41-65a4-43d1-92a3-d9e839e3265f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.461200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-1659df41-65a4-43d1-92a3-d9e839e3265f None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:44.462458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-1659df41-65a4-43d1-92a3-d9e839e3265f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:44.489074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03b9f8-b569-47d9-8b71-cff2e90d5527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:44.489764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03b9f8-b569-47d9-8b71-cff2e90d5527 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.490358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03b9f8-b569-47d9-8b71-cff2e90d5527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.490358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03b9f8-b569-47d9-8b71-cff2e90d5527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.491102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03b9f8-b569-47d9-8b71-cff2e90d5527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:44.492947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03b9f8-b569-47d9-8b71-cff2e90d5527 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:44.503801 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1178/2354] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:44.509310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03b9f8-b569-47d9-8b71-cff2e90d5527 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:44.511483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9d03b9f8-b569-47d9-8b71-cff2e90d5527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:44.525281 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03b9f8-b569-47d9-8b71-cff2e90d5527 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:44.530423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1177/2355] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:44.530589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256e585-fbd2-4874-b07c-321eca5aa52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:44.531057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e585-fbd2-4874-b07c-321eca5aa52b admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.531413 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e585-fbd2-4874-b07c-321eca5aa52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.531760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e585-fbd2-4874-b07c-321eca5aa52b admin admin] PATH_INFO: `/v3/projects/acffa38375ba4b9b8d9b59cb1a933156/users/8fb3c5cf57f54d1c90ed1ea6088ea1fa/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.538568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcc6e9-4db3-46a2-95a6-9d886898ae9a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.538873 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcc6e9-4db3-46a2-95a6-9d886898ae9a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.539705 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcc6e9-4db3-46a2-95a6-9d886898ae9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.549053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256e585-fbd2-4874-b07c-321eca5aa52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=acffa38375ba4b9b8d9b59cb1a933156, user_id=8fb3c5cf57f54d1c90ed1ea6088ea1fa, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:44.551917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256e585-fbd2-4874-b07c-321eca5aa52b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:44.558466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-45bcc6e9-4db3-46a2-95a6-9d886898ae9a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:44.559741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-45bcc6e9-4db3-46a2-95a6-9d886898ae9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:44.567747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1178/2356] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:44.572141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcc6e9-4db3-46a2-95a6-9d886898ae9a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:44.581176 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcc6e9-4db3-46a2-95a6-9d886898ae9a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:44.587126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972cbb5-4065-4499-9bb1-3f59d6c18b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:44.587721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972cbb5-4065-4499-9bb1-3f59d6c18b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.587998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972cbb5-4065-4499-9bb1-3f59d6c18b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.588266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972cbb5-4065-4499-9bb1-3f59d6c18b4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.588942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972cbb5-4065-4499-9bb1-3f59d6c18b4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:23:44.589244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972cbb5-4065-4499-9bb1-3f59d6c18b4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:44.591046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972cbb5-4065-4499-9bb1-3f59d6c18b4e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:44.607690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1179/2357] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:23:44.612338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972cbb5-4065-4499-9bb1-3f59d6c18b4e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:44.622983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972cbb5-4065-4499-9bb1-3f59d6c18b4e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:44.627751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3242b-c1cc-4115-8bd4-6c648855e896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:44.628348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3242b-c1cc-4115-8bd4-6c648855e896 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.628626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3242b-c1cc-4115-8bd4-6c648855e896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.628930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3242b-c1cc-4115-8bd4-6c648855e896 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.629550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3242b-c1cc-4115-8bd4-6c648855e896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:44.631439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3242b-c1cc-4115-8bd4-6c648855e896 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:44.657463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1179/2358] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:44 2022] PUT /identity/v3/projects/acffa38375ba4b9b8d9b59cb1a933156/users/8fb3c5cf57f54d1c90ed1ea6088ea1fa/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:44.666908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256e585-fbd2-4874-b07c-321eca5aa52b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:44.729818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256e585-fbd2-4874-b07c-321eca5aa52b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:44.736247 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bdd61-7b04-45a0-9202-6d5ce297f50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:44.738960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bdd61-7b04-45a0-9202-6d5ce297f50c admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.738960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bdd61-7b04-45a0-9202-6d5ce297f50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.738960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bdd61-7b04-45a0-9202-6d5ce297f50c admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.738960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59bdd61-7b04-45a0-9202-6d5ce297f50c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:44.743379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59bdd61-7b04-45a0-9202-6d5ce297f50c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:44.743571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1180/2359] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:44 2022] POST /identity/v3/projects => generated 403 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:44.744754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3242b-c1cc-4115-8bd4-6c648855e896 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:44.751662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1180/2360] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:44.755735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bdd61-7b04-45a0-9202-6d5ce297f50c admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:44.767312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bdd61-7b04-45a0-9202-6d5ce297f50c admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:44.826179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3242b-c1cc-4115-8bd4-6c648855e896 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:44.833062 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:44.835722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b9378-1193-4466-8a47-0abea20f5b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:44.836691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b9378-1193-4466-8a47-0abea20f5b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.837126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b9378-1193-4466-8a47-0abea20f5b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.837529 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b9378-1193-4466-8a47-0abea20f5b49 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.839449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b9378-1193-4466-8a47-0abea20f5b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:44.842704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b9378-1193-4466-8a47-0abea20f5b49 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:44.892763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb63ed-9e5a-4ee0-8604-0f8967a07389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:44.893573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb63ed-9e5a-4ee0-8604-0f8967a07389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.901445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb63ed-9e5a-4ee0-8604-0f8967a07389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.901892 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb63ed-9e5a-4ee0-8604-0f8967a07389 admin admin] PATH_INFO: `/v3/projects/acffa38375ba4b9b8d9b59cb1a933156/users/8fb3c5cf57f54d1c90ed1ea6088ea1fa/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.904692 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb63ed-9e5a-4ee0-8604-0f8967a07389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acffa38375ba4b9b8d9b59cb1a933156, user_id=8fb3c5cf57f54d1c90ed1ea6088ea1fa, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:44.907987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb63ed-9e5a-4ee0-8604-0f8967a07389 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:44.925981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1181/2361] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:44 2022] PUT /identity/v3/projects/acffa38375ba4b9b8d9b59cb1a933156/users/8fb3c5cf57f54d1c90ed1ea6088ea1fa/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:44.938116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c506e-60a5-4639-a668-847256daa8c9 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:44.938362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c506e-60a5-4639-a668-847256daa8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:44.938640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c506e-60a5-4639-a668-847256daa8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:44.973379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-cb2c506e-60a5-4639-a668-847256daa8c9 None None] MFA Rules not processed for user `8fb3c5cf57f54d1c90ed1ea6088ea1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:44.975123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-cb2c506e-60a5-4639-a668-847256daa8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:45.016493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1181/2362] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:44 2022] POST /identity/v3/users => generated 302 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:45.025208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b9378-1193-4466-8a47-0abea20f5b49 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:45.041223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b9378-1193-4466-8a47-0abea20f5b49 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:45.059689 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1182/2363] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:45.075003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c506e-60a5-4639-a668-847256daa8c9 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:45.076899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-cb2c506e-60a5-4639-a668-847256daa8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:45.108232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f90e3-c7cc-4a07-a6d2-a92875f84527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:45.109055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f90e3-c7cc-4a07-a6d2-a92875f84527 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:45.109549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f90e3-c7cc-4a07-a6d2-a92875f84527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:45.109930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f90e3-c7cc-4a07-a6d2-a92875f84527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:45.110966 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f90e3-c7cc-4a07-a6d2-a92875f84527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:45.112928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f90e3-c7cc-4a07-a6d2-a92875f84527 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:45.126653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1182/2364] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:45.130775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f90e3-c7cc-4a07-a6d2-a92875f84527 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:45.140469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c506e-60a5-4639-a668-847256daa8c9 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:45.143283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f90e3-c7cc-4a07-a6d2-a92875f84527 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:45.144634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecdc95-5d97-4243-a376-7173452a362a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:45.145280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecdc95-5d97-4243-a376-7173452a362a service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:45.145555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecdc95-5d97-4243-a376-7173452a362a service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:45.145824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecdc95-5d97-4243-a376-7173452a362a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:45.149887 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602dc1-4b25-4613-9fc5-134f87d964c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:45.150859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602dc1-4b25-4613-9fc5-134f87d964c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:45.151449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602dc1-4b25-4613-9fc5-134f87d964c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:45.151939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602dc1-4b25-4613-9fc5-134f87d964c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:45.152885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2602dc1-4b25-4613-9fc5-134f87d964c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:45.154834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2602dc1-4b25-4613-9fc5-134f87d964c8 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:45.159923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecdc95-5d97-4243-a376-7173452a362a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:45.161394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecdc95-5d97-4243-a376-7173452a362a service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:45.168282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1183/2365] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:45.172548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602dc1-4b25-4613-9fc5-134f87d964c8 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:45.173547 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1183/2366] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:23:45.186939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:45.188397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602dc1-4b25-4613-9fc5-134f87d964c8 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:45.193869 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cc8d1-eb39-48b4-a215-eea125cb80ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:45.195149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cc8d1-eb39-48b4-a215-eea125cb80ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:45.195838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cc8d1-eb39-48b4-a215-eea125cb80ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:45.196128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cc8d1-eb39-48b4-a215-eea125cb80ef admin admin] PATH_INFO: `/v3/projects/061a17d5e3cd498f9f813855420c6b97/users/e6f024bf7bb34e3d81aeac7d8bf9eb93/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:45.217343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cc8d1-eb39-48b4-a215-eea125cb80ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=061a17d5e3cd498f9f813855420c6b97, user_id=e6f024bf7bb34e3d81aeac7d8bf9eb93, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:45.220060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cc8d1-eb39-48b4-a215-eea125cb80ef admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:45.243963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecdc95-5d97-4243-a376-7173452a362a service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:45.256909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecdc95-5d97-4243-a376-7173452a362a service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:45.262069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc4637-d357-4ec0-91cb-02bdb97aaca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:45.264906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4637-d357-4ec0-91cb-02bdb97aaca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:45.264906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4637-d357-4ec0-91cb-02bdb97aaca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:45.264906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4637-d357-4ec0-91cb-02bdb97aaca3 admin admin] PATH_INFO: `/v3/users/e765c4b152d340fe8bd4cca003d4c446` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:45.274108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc4637-d357-4ec0-91cb-02bdb97aaca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e765c4b152d340fe8bd4cca003d4c446)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:45.277592 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc4637-d357-4ec0-91cb-02bdb97aaca3 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:45.366853 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1184/2367] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:45 2022] PUT /identity/v3/projects/061a17d5e3cd498f9f813855420c6b97/users/e6f024bf7bb34e3d81aeac7d8bf9eb93/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:45.372040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec194f-81e6-4a61-a474-e245e4f7489e None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:45.375342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec194f-81e6-4a61-a474-e245e4f7489e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:45.375342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec194f-81e6-4a61-a474-e245e4f7489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:45.395302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-e6ec194f-81e6-4a61-a474-e245e4f7489e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:45.396742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e6ec194f-81e6-4a61-a474-e245e4f7489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:45.398476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:45.464717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1185/2368] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:45.466013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024d6ef-b4c2-45b3-bc0a-3163bbad4abc None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:45.466413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024d6ef-b4c2-45b3-bc0a-3163bbad4abc None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:45.466858 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024d6ef-b4c2-45b3-bc0a-3163bbad4abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:45.489094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-f024d6ef-b4c2-45b3-bc0a-3163bbad4abc None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:45.490387 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f024d6ef-b4c2-45b3-bc0a-3163bbad4abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:45.527963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7ebc4637-d357-4ec0-91cb-02bdb97aaca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e765c4b152d340fe8bd4cca003d4c446'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:45.529232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7ebc4637-d357-4ec0-91cb-02bdb97aaca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e765c4b152d340fe8bd4cca003d4c446'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:45.537674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7ebc4637-d357-4ec0-91cb-02bdb97aaca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e765c4b152d340fe8bd4cca003d4c446'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:45.541064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7ebc4637-d357-4ec0-91cb-02bdb97aaca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e765c4b152d340fe8bd4cca003d4c446'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:45.553002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1186/2369] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:23:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:45.553989 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024d6ef-b4c2-45b3-bc0a-3163bbad4abc None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:45.555511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f024d6ef-b4c2-45b3-bc0a-3163bbad4abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:45.614418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024d6ef-b4c2-45b3-bc0a-3163bbad4abc None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:45.618310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d613393-b129-43a4-bada-21599374c122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:45.618961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d613393-b129-43a4-bada-21599374c122 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:45.619296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d613393-b129-43a4-bada-21599374c122 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:45.619898 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d613393-b129-43a4-bada-21599374c122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:45.621718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8d613393-b129-43a4-bada-21599374c122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:45.634445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d613393-b129-43a4-bada-21599374c122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:45.635869 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d613393-b129-43a4-bada-21599374c122 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:45.645617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1187/2370] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:45.645776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7ebc4637-d357-4ec0-91cb-02bdb97aaca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e765c4b152d340fe8bd4cca003d4c446'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:45.648064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d613393-b129-43a4-bada-21599374c122 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:45.648731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8d613393-b129-43a4-bada-21599374c122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:45.670987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1184/2371] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:45 2022] DELETE /identity/v3/users/e765c4b152d340fe8bd4cca003d4c446 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:45.672883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd26525-1aba-47e0-bf8a-7eafdd7fa08c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:45.673135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd26525-1aba-47e0-bf8a-7eafdd7fa08c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:45.673432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd26525-1aba-47e0-bf8a-7eafdd7fa08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:45.694850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-0bd26525-1aba-47e0-bf8a-7eafdd7fa08c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:45.696175 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0bd26525-1aba-47e0-bf8a-7eafdd7fa08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:45.719166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d613393-b129-43a4-bada-21599374c122 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:45.758314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1185/2372] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:45.768203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd26525-1aba-47e0-bf8a-7eafdd7fa08c None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:45.782443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dafc0f-9476-4ea5-8efb-724e3b4c3fcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:45.783146 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dafc0f-9476-4ea5-8efb-724e3b4c3fcb service swift] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:45.783500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dafc0f-9476-4ea5-8efb-724e3b4c3fcb service swift] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:45.783846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dafc0f-9476-4ea5-8efb-724e3b4c3fcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:45.826203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd26525-1aba-47e0-bf8a-7eafdd7fa08c None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:45.829896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb32829-e7a8-4451-9823-bcfa4b9f5a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:45.830513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb32829-e7a8-4451-9823-bcfa4b9f5a1f service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:45.830793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb32829-e7a8-4451-9823-bcfa4b9f5a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:45.831069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb32829-e7a8-4451-9823-bcfa4b9f5a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:45.843601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb32829-e7a8-4451-9823-bcfa4b9f5a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:45.845157 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb32829-e7a8-4451-9823-bcfa4b9f5a1f service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:45.856523 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1186/2373] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:45.862364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dafc0f-9476-4ea5-8efb-724e3b4c3fcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:45.864006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dafc0f-9476-4ea5-8efb-724e3b4c3fcb service swift] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:45.873695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1188/2374] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:23:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:23:45.954457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:46.103352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:46.719518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb32829-e7a8-4451-9823-bcfa4b9f5a1f service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:46.721267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ebb32829-e7a8-4451-9823-bcfa4b9f5a1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:46.752777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb32829-e7a8-4451-9823-bcfa4b9f5a1f service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:46.774478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:46.776058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:46.776501 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:46.777048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] PATH_INFO: `/v3/projects/44b040511d164d2b9d0061c8dc80e4ca` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:46.779062 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b040511d164d2b9d0061c8dc80e4ca)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:46.784477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:47.052270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b040511d164d2b9d0061c8dc80e4ca', 'request_id': 'req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:47.058109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:47.063876 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b040511d164d2b9d0061c8dc80e4ca', 'request_id': 'req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:47.072780 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b040511d164d2b9d0061c8dc80e4ca', 'request_id': 'req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:47.206466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] The token cache is being invalidate because project 44b040511d164d2b9d0061c8dc80e4ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:47.206763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:47.207432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-c856e4a5-fbeb-4cdf-a691-7ee2e91857c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:47.209172 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1187/2375] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:46 2022] DELETE /identity/v3/projects/44b040511d164d2b9d0061c8dc80e4ca => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:47.213692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dafc0f-9476-4ea5-8efb-724e3b4c3fcb service swift] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:47.251345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dafc0f-9476-4ea5-8efb-724e3b4c3fcb service swift] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:47.304966 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:47.375166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:47.375978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:47.376333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:47.376738 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] PATH_INFO: `/v3/users/8fb3c5cf57f54d1c90ed1ea6088ea1fa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:47.378055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb3c5cf57f54d1c90ed1ea6088ea1fa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:47.394540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:47.590765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb8100-baf0-4d05-baea-b0fb625415c9 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:47.591060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb8100-baf0-4d05-baea-b0fb625415c9 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:47.591652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb8100-baf0-4d05-baea-b0fb625415c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:47.644146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1afb8100-baf0-4d05-baea-b0fb625415c9 None None] MFA Rules not processed for user `e6f024bf7bb34e3d81aeac7d8bf9eb93`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:47.645670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1afb8100-baf0-4d05-baea-b0fb625415c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:47.684046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb3c5cf57f54d1c90ed1ea6088ea1fa'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:47.686099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb3c5cf57f54d1c90ed1ea6088ea1fa'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:47.695560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb3c5cf57f54d1c90ed1ea6088ea1fa'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:47.709013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb3c5cf57f54d1c90ed1ea6088ea1fa'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:47.766340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1188/2376] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:47 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:47.771261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb8100-baf0-4d05-baea-b0fb625415c9 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:47.773418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1afb8100-baf0-4d05-baea-b0fb625415c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:47.789773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb8100-baf0-4d05-baea-b0fb625415c9 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:47.795931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26698a47-bf0d-4932-bd10-c5907bc821b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:47.796562 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26698a47-bf0d-4932-bd10-c5907bc821b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:47.796945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26698a47-bf0d-4932-bd10-c5907bc821b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:47.797233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26698a47-bf0d-4932-bd10-c5907bc821b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:47.797872 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26698a47-bf0d-4932-bd10-c5907bc821b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:47.799794 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26698a47-bf0d-4932-bd10-c5907bc821b3 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:47.810805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb3c5cf57f54d1c90ed1ea6088ea1fa'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:47.825630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1189/2377] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:47 2022] DELETE /identity/v3/users/8fb3c5cf57f54d1c90ed1ea6088ea1fa => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:47.941286 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1189/2378] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:47 2022] POST /identity/v3/projects => generated 403 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:47.946298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:47.948210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:48.052226 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec18f9-785c-4343-9fd4-b3962bbebd44 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:48.108023 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:48.128815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c53c6-f8d7-47f6-9a96-02b21e6ff997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:48.129482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c53c6-f8d7-47f6-9a96-02b21e6ff997 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:48.129917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c53c6-f8d7-47f6-9a96-02b21e6ff997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:48.130110 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c53c6-f8d7-47f6-9a96-02b21e6ff997 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:48.130766 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c53c6-f8d7-47f6-9a96-02b21e6ff997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:48.133547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c53c6-f8d7-47f6-9a96-02b21e6ff997 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:48.281282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1190/2379] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:47 2022] POST /identity/v3/users => generated 308 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:48.285897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26698a47-bf0d-4932-bd10-c5907bc821b3 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:48.321581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26698a47-bf0d-4932-bd10-c5907bc821b3 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:48.354164 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b39f6f-573e-449a-bdec-7b6155825f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:48.356003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39f6f-573e-449a-bdec-7b6155825f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:48.356700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39f6f-573e-449a-bdec-7b6155825f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:48.357271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39f6f-573e-449a-bdec-7b6155825f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:48.364125 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b39f6f-573e-449a-bdec-7b6155825f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:48.369448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b39f6f-573e-449a-bdec-7b6155825f10 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:48.382536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1190/2380] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:48.389709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c53c6-f8d7-47f6-9a96-02b21e6ff997 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:48.404544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c53c6-f8d7-47f6-9a96-02b21e6ff997 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:48.410699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581fb4c-2632-4c5c-be39-7dc2c92f5f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:48.413022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581fb4c-2632-4c5c-be39-7dc2c92f5f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:48.413022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581fb4c-2632-4c5c-be39-7dc2c92f5f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:48.413022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581fb4c-2632-4c5c-be39-7dc2c92f5f1f admin admin] PATH_INFO: `/v3/projects/89810f71039342739650be12b7a8037d/users/41fe5ea6287e4cc18659066f10e83e72/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:48.431167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581fb4c-2632-4c5c-be39-7dc2c92f5f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=89810f71039342739650be12b7a8037d, user_id=41fe5ea6287e4cc18659066f10e83e72, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:48.434332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581fb4c-2632-4c5c-be39-7dc2c92f5f1f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:48.532594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1191/2381] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:48 2022] PUT /identity/v3/projects/89810f71039342739650be12b7a8037d/users/41fe5ea6287e4cc18659066f10e83e72/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:48.536988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b39f6f-573e-449a-bdec-7b6155825f10 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:48.555366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b39f6f-573e-449a-bdec-7b6155825f10 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:48.580949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581fb4c-2632-4c5c-be39-7dc2c92f5f1f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:48.583307 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8581fb4c-2632-4c5c-be39-7dc2c92f5f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:48.652055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:48.659878 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:48.742109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843989f2-7c1c-4d92-be41-d0ad251f83c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:48.743019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843989f2-7c1c-4d92-be41-d0ad251f83c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:48.743301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843989f2-7c1c-4d92-be41-d0ad251f83c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:48.743574 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843989f2-7c1c-4d92-be41-d0ad251f83c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:48.744388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843989f2-7c1c-4d92-be41-d0ad251f83c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:48.753632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581fb4c-2632-4c5c-be39-7dc2c92f5f1f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:48.761220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843989f2-7c1c-4d92-be41-d0ad251f83c4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:48.769125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:48.769885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:48.770300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:48.770503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] PATH_INFO: `/v3/projects/acffa38375ba4b9b8d9b59cb1a933156` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:48.772214 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] RBAC: Authorizing `identity:delete_project(project_id=acffa38375ba4b9b8d9b59cb1a933156)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:48.774622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:48.782127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1191/2382] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:48.790156 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843989f2-7c1c-4d92-be41-d0ad251f83c4 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:48.811338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843989f2-7c1c-4d92-be41-d0ad251f83c4 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:48.818576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3de1f-04de-49bf-a037-34f680566826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:48.819785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3de1f-04de-49bf-a037-34f680566826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:48.820592 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3de1f-04de-49bf-a037-34f680566826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:48.821148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3de1f-04de-49bf-a037-34f680566826 admin admin] PATH_INFO: `/v3/projects/89810f71039342739650be12b7a8037d/users/41fe5ea6287e4cc18659066f10e83e72/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:48.823681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3de1f-04de-49bf-a037-34f680566826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89810f71039342739650be12b7a8037d, user_id=41fe5ea6287e4cc18659066f10e83e72, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:48.826731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3de1f-04de-49bf-a037-34f680566826 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:48.838223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1192/2383] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:48 2022] PUT /identity/v3/projects/89810f71039342739650be12b7a8037d/users/41fe5ea6287e4cc18659066f10e83e72/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:48.934184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acffa38375ba4b9b8d9b59cb1a933156', 'request_id': 'req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:48.943513 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acffa38375ba4b9b8d9b59cb1a933156', 'request_id': 'req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:49.040206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acffa38375ba4b9b8d9b59cb1a933156', 'request_id': 'req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:49.060975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] The token cache is being invalidate because project acffa38375ba4b9b8d9b59cb1a933156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:49.061939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:49.061939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b5c3f41c-3f39-4f09-bb8e-1eadd235d06d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:49.063533 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1192/2384] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:48 2022] DELETE /identity/v3/projects/acffa38375ba4b9b8d9b59cb1a933156 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:49.069628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe27b5e-ad50-4680-b435-df4e475024a3 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:49.069920 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe27b5e-ad50-4680-b435-df4e475024a3 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:49.070249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe27b5e-ad50-4680-b435-df4e475024a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:49.124659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-4fe27b5e-ad50-4680-b435-df4e475024a3 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:49.130268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-4fe27b5e-ad50-4680-b435-df4e475024a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:49.287813 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1193/2385] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:49.301450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852530c5-4c6a-4b0f-9280-c6fc1641a346 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:49.302291 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852530c5-4c6a-4b0f-9280-c6fc1641a346 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:49.302859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852530c5-4c6a-4b0f-9280-c6fc1641a346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:49.355396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-852530c5-4c6a-4b0f-9280-c6fc1641a346 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:49.357091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-852530c5-4c6a-4b0f-9280-c6fc1641a346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:49.367301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1193/2386] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:49.371859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe27b5e-ad50-4680-b435-df4e475024a3 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:49.384105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe27b5e-ad50-4680-b435-df4e475024a3 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:49.395401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34daae86-3f64-4723-b81d-f888c45a0345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:49.395843 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daae86-3f64-4723-b81d-f888c45a0345 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:49.397749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daae86-3f64-4723-b81d-f888c45a0345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:49.405340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daae86-3f64-4723-b81d-f888c45a0345 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:49.406496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34daae86-3f64-4723-b81d-f888c45a0345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:23:49.407143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34daae86-3f64-4723-b81d-f888c45a0345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:49.409509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34daae86-3f64-4723-b81d-f888c45a0345 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:49.443207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1194/2387] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:23:49.449665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852530c5-4c6a-4b0f-9280-c6fc1641a346 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:49.468243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852530c5-4c6a-4b0f-9280-c6fc1641a346 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:49.485986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150a598-5c29-46e1-a0c4-4c935fedef01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:49.487178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a598-5c29-46e1-a0c4-4c935fedef01 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:49.487178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a598-5c29-46e1-a0c4-4c935fedef01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:49.487178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a598-5c29-46e1-a0c4-4c935fedef01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:49.488067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150a598-5c29-46e1-a0c4-4c935fedef01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:49.490168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150a598-5c29-46e1-a0c4-4c935fedef01 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:49.653155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1194/2388] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:49 2022] POST /identity/v3/projects => generated 391 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:49.656192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34daae86-3f64-4723-b81d-f888c45a0345 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:49.670252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34daae86-3f64-4723-b81d-f888c45a0345 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:49.743215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb21fe-2d98-4a4e-bca7-a41852e3f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:49.744155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb21fe-2d98-4a4e-bca7-a41852e3f772 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:49.744433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb21fe-2d98-4a4e-bca7-a41852e3f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:49.744760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb21fe-2d98-4a4e-bca7-a41852e3f772 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:49.745431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb21fe-2d98-4a4e-bca7-a41852e3f772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:49.749082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb21fe-2d98-4a4e-bca7-a41852e3f772 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:49.871684 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1195/2389] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:49 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:49.875544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150a598-5c29-46e1-a0c4-4c935fedef01 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:49.895016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150a598-5c29-46e1-a0c4-4c935fedef01 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:49.901377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec76691-161f-4017-861b-c53d00729d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:49.902098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec76691-161f-4017-861b-c53d00729d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:49.902384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec76691-161f-4017-861b-c53d00729d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:49.902660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec76691-161f-4017-861b-c53d00729d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:49.903316 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec76691-161f-4017-861b-c53d00729d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:49.905170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec76691-161f-4017-861b-c53d00729d81 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:49.927421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1195/2390] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:49.932222 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb21fe-2d98-4a4e-bca7-a41852e3f772 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:49.950637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb21fe-2d98-4a4e-bca7-a41852e3f772 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:49.956873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc5658-1c4b-4190-89aa-f73acc779617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:49.957665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc5658-1c4b-4190-89aa-f73acc779617 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:49.957955 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc5658-1c4b-4190-89aa-f73acc779617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:49.958246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc5658-1c4b-4190-89aa-f73acc779617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:49.958974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc5658-1c4b-4190-89aa-f73acc779617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:49.961420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc5658-1c4b-4190-89aa-f73acc779617 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:49.978525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1196/2391] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:49.983002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec76691-161f-4017-861b-c53d00729d81 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:49.999816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec76691-161f-4017-861b-c53d00729d81 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:50.017279 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfd180-bd2b-431b-9552-63c0a3435607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:50.019096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfd180-bd2b-431b-9552-63c0a3435607 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:50.019681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfd180-bd2b-431b-9552-63c0a3435607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:50.020030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfd180-bd2b-431b-9552-63c0a3435607 admin admin] PATH_INFO: `/v3/projects/712466a083bd40d89bc3866c13ad4930/users/a2f6a1da30a64131a41e5d5fa3883d5d/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:50.052407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfd180-bd2b-431b-9552-63c0a3435607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=712466a083bd40d89bc3866c13ad4930, user_id=a2f6a1da30a64131a41e5d5fa3883d5d, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:50.056266 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfd180-bd2b-431b-9552-63c0a3435607 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:50.207245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1196/2392] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:49 2022] PUT /identity/v3/projects/712466a083bd40d89bc3866c13ad4930/users/a2f6a1da30a64131a41e5d5fa3883d5d/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:50.218214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929e431-627a-4c6a-9be8-7b724e99b198 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:50.218506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929e431-627a-4c6a-9be8-7b724e99b198 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:50.218782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929e431-627a-4c6a-9be8-7b724e99b198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:50.259420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6929e431-627a-4c6a-9be8-7b724e99b198 None None] MFA Rules not processed for user `a2f6a1da30a64131a41e5d5fa3883d5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:50.260986 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6929e431-627a-4c6a-9be8-7b724e99b198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:50.271650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:50.347155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1197/2393] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:50.348693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfd180-bd2b-431b-9552-63c0a3435607 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:50.357847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:50.463603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfd180-bd2b-431b-9552-63c0a3435607 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:50.468557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e980d-75d6-4cc6-898b-86e3cdac0d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:50.469213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e980d-75d6-4cc6-898b-86e3cdac0d61 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:50.469494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e980d-75d6-4cc6-898b-86e3cdac0d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:50.469771 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e980d-75d6-4cc6-898b-86e3cdac0d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:50.470453 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e980d-75d6-4cc6-898b-86e3cdac0d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:50.472615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e980d-75d6-4cc6-898b-86e3cdac0d61 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:50.620466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929e431-627a-4c6a-9be8-7b724e99b198 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:50.622063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1197/2394] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:50 2022] POST /identity/v3/projects => generated 393 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:50.628844 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6929e431-627a-4c6a-9be8-7b724e99b198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:50.632661 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e980d-75d6-4cc6-898b-86e3cdac0d61 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:50.722141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929e431-627a-4c6a-9be8-7b724e99b198 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:50.760172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e980d-75d6-4cc6-898b-86e3cdac0d61 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:50.769216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522146ad-ee84-4305-b6e2-ca25c8ebbf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:50.770000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522146ad-ee84-4305-b6e2-ca25c8ebbf9d admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:50.770293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522146ad-ee84-4305-b6e2-ca25c8ebbf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:50.770573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522146ad-ee84-4305-b6e2-ca25c8ebbf9d admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:50.771382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522146ad-ee84-4305-b6e2-ca25c8ebbf9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:50.773781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522146ad-ee84-4305-b6e2-ca25c8ebbf9d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:50.800319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bd8ae-646e-492e-8a9c-4a832d304e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:50.801103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bd8ae-646e-492e-8a9c-4a832d304e3b service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:50.801447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bd8ae-646e-492e-8a9c-4a832d304e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:50.801814 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bd8ae-646e-492e-8a9c-4a832d304e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:50.807815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6d0bd8ae-646e-492e-8a9c-4a832d304e3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:50.822421 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bd8ae-646e-492e-8a9c-4a832d304e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:50.824032 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bd8ae-646e-492e-8a9c-4a832d304e3b service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:50.850083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1198/2395] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:23:50.850083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562033e5-7044-4a6d-b39a-dfe28bd61be0 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:50.850083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562033e5-7044-4a6d-b39a-dfe28bd61be0 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:50.850083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562033e5-7044-4a6d-b39a-dfe28bd61be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:50.887657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1198/2396] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:50 2022] POST /identity/v3/users => generated 303 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:50.891799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522146ad-ee84-4305-b6e2-ca25c8ebbf9d admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:50.901414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-562033e5-7044-4a6d-b39a-dfe28bd61be0 None None] MFA Rules not processed for user `41fe5ea6287e4cc18659066f10e83e72`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:50.905562 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-562033e5-7044-4a6d-b39a-dfe28bd61be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:50.909150 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522146ad-ee84-4305-b6e2-ca25c8ebbf9d admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:50.922131 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642f765-5588-48ef-bfc4-d83d38161c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:50.922878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642f765-5588-48ef-bfc4-d83d38161c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:50.923276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642f765-5588-48ef-bfc4-d83d38161c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:50.923565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642f765-5588-48ef-bfc4-d83d38161c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:50.924551 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642f765-5588-48ef-bfc4-d83d38161c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:50.927850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642f765-5588-48ef-bfc4-d83d38161c4b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:50.964614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1199/2397] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:50.968120 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642f765-5588-48ef-bfc4-d83d38161c4b admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:50.988486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642f765-5588-48ef-bfc4-d83d38161c4b admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:51.001352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb623ed-820f-4a52-9907-7e1e0f120bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:51.002028 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb623ed-820f-4a52-9907-7e1e0f120bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:51.002495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb623ed-820f-4a52-9907-7e1e0f120bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:51.002783 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb623ed-820f-4a52-9907-7e1e0f120bd2 admin admin] PATH_INFO: `/v3/projects/c45a7c273e3f43d7a2eefe342074cff8/users/461ef58840464370bef92620d335615e/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:51.007005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:51.046111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb623ed-820f-4a52-9907-7e1e0f120bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45a7c273e3f43d7a2eefe342074cff8, user_id=461ef58840464370bef92620d335615e, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:51.046840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1199/2398] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:51.049136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb623ed-820f-4a52-9907-7e1e0f120bd2 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:51.224367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1200/2399] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:50 2022] PUT /identity/v3/projects/c45a7c273e3f43d7a2eefe342074cff8/users/461ef58840464370bef92620d335615e/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:51.229762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562033e5-7044-4a6d-b39a-dfe28bd61be0 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:51.253620 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562033e5-7044-4a6d-b39a-dfe28bd61be0 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:51.383783 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7b8e8-1cfd-48a1-8f04-fa17b7d48ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:51.384408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b8e8-1cfd-48a1-8f04-fa17b7d48ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:51.384715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b8e8-1cfd-48a1-8f04-fa17b7d48ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:51.385726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b8e8-1cfd-48a1-8f04-fa17b7d48ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:51.387330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7b8e8-1cfd-48a1-8f04-fa17b7d48ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:51.391467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7b8e8-1cfd-48a1-8f04-fa17b7d48ef9 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:51.422194 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1200/2400] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:51.426426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb623ed-820f-4a52-9907-7e1e0f120bd2 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:51.445196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb623ed-820f-4a52-9907-7e1e0f120bd2 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:51.455856 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ff5ee-c10d-49c4-8cb7-1edf0aa775e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:51.456765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ff5ee-c10d-49c4-8cb7-1edf0aa775e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:51.457224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ff5ee-c10d-49c4-8cb7-1edf0aa775e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:51.457679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ff5ee-c10d-49c4-8cb7-1edf0aa775e9 admin admin] PATH_INFO: `/v3/projects/c45a7c273e3f43d7a2eefe342074cff8/users/461ef58840464370bef92620d335615e/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:51.461091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ff5ee-c10d-49c4-8cb7-1edf0aa775e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45a7c273e3f43d7a2eefe342074cff8, user_id=461ef58840464370bef92620d335615e, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:51.467102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ff5ee-c10d-49c4-8cb7-1edf0aa775e9 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:51.493229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1201/2401] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:51 2022] PUT /identity/v3/projects/c45a7c273e3f43d7a2eefe342074cff8/users/461ef58840464370bef92620d335615e/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:51.494887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070991e-bcb4-49da-bc9c-b12c1e4f7dfd None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:51.495656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070991e-bcb4-49da-bc9c-b12c1e4f7dfd None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:51.495656 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070991e-bcb4-49da-bc9c-b12c1e4f7dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:51.551273 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e070991e-bcb4-49da-bc9c-b12c1e4f7dfd None None] MFA Rules not processed for user `461ef58840464370bef92620d335615e`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:51.552531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e070991e-bcb4-49da-bc9c-b12c1e4f7dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:51.739179 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1201/2402] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:51 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:51.751591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ff5ee-c10d-49c4-8cb7-1edf0aa775e9 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:51.753488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-894ff5ee-c10d-49c4-8cb7-1edf0aa775e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:51.829849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:51.863607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ff5ee-c10d-49c4-8cb7-1edf0aa775e9 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:51.869640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d1b3f-f36b-49b8-9302-628bc2382a95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:51.870545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1b3f-f36b-49b8-9302-628bc2382a95 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:51.871497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1b3f-f36b-49b8-9302-628bc2382a95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:51.871979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1b3f-f36b-49b8-9302-628bc2382a95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:51.884447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070991e-bcb4-49da-bc9c-b12c1e4f7dfd None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:51.889398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d1b3f-f36b-49b8-9302-628bc2382a95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:51.889527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e070991e-bcb4-49da-bc9c-b12c1e4f7dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:51.889626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d1b3f-f36b-49b8-9302-628bc2382a95 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:51.900567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1202/2403] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:51 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:51.977064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d1b3f-f36b-49b8-9302-628bc2382a95 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:51.979250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f73d1b3f-f36b-49b8-9302-628bc2382a95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:52.002505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d1b3f-f36b-49b8-9302-628bc2382a95 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:52.002505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763cf3d-fc16-4816-a1b3-689a14c9bd43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:52.002505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763cf3d-fc16-4816-a1b3-689a14c9bd43 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:52.002505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763cf3d-fc16-4816-a1b3-689a14c9bd43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:52.003145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763cf3d-fc16-4816-a1b3-689a14c9bd43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:52.082417 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070991e-bcb4-49da-bc9c-b12c1e4f7dfd None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:52.098343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6710e1f-542a-4452-99d4-ab8b84e6f1ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:52.099409 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6710e1f-542a-4452-99d4-ab8b84e6f1ec service placement] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:52.101855 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6710e1f-542a-4452-99d4-ab8b84e6f1ec service placement] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:52.102918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6710e1f-542a-4452-99d4-ab8b84e6f1ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:52.106526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763cf3d-fc16-4816-a1b3-689a14c9bd43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:52.113074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c6710e1f-542a-4452-99d4-ab8b84e6f1ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:52.118383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763cf3d-fc16-4816-a1b3-689a14c9bd43 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:52.143675 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1203/2404] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:52.155861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6710e1f-542a-4452-99d4-ab8b84e6f1ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:52.159603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6710e1f-542a-4452-99d4-ab8b84e6f1ec service placement] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:52.191346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1202/2405] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:53.266247 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763cf3d-fc16-4816-a1b3-689a14c9bd43 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:53.267754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6763cf3d-fc16-4816-a1b3-689a14c9bd43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:53.425507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763cf3d-fc16-4816-a1b3-689a14c9bd43 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:53.434054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:53.441730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:53.442397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:53.442982 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] PATH_INFO: `/v3/users/e6f024bf7bb34e3d81aeac7d8bf9eb93` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:53.453587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f024bf7bb34e3d81aeac7d8bf9eb93)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:53.471488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:54.140044 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f024bf7bb34e3d81aeac7d8bf9eb93'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:54.141200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f024bf7bb34e3d81aeac7d8bf9eb93'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:54.157581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f024bf7bb34e3d81aeac7d8bf9eb93'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:54.198276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f024bf7bb34e3d81aeac7d8bf9eb93'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:54.200017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:54.331294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f024bf7bb34e3d81aeac7d8bf9eb93'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:54.338091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1204/2406] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:53 2022] DELETE /identity/v3/users/e6f024bf7bb34e3d81aeac7d8bf9eb93 => generated 0 bytes in 1073 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:54.862079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6710e1f-542a-4452-99d4-ab8b84e6f1ec service placement] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:54.868433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:54.876008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6710e1f-542a-4452-99d4-ab8b84e6f1ec service placement] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:55.013930 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:55.014597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:55.014881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:55.015158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] PATH_INFO: `/v3/projects/061a17d5e3cd498f9f813855420c6b97` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:55.017046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=061a17d5e3cd498f9f813855420c6b97)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:55.020821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:56.008256 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061a17d5e3cd498f9f813855420c6b97', 'request_id': 'req-e5096849-cac3-4a65-bd04-d992ee23b433'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:56.013570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061a17d5e3cd498f9f813855420c6b97', 'request_id': 'req-e5096849-cac3-4a65-bd04-d992ee23b433'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:56.020199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061a17d5e3cd498f9f813855420c6b97', 'request_id': 'req-e5096849-cac3-4a65-bd04-d992ee23b433'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:56.142828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] The token cache is being invalidate because project 061a17d5e3cd498f9f813855420c6b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:56.150248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:56.150248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:56.150248 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1203/2407] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:54 2022] DELETE /identity/v3/projects/061a17d5e3cd498f9f813855420c6b97 => generated 0 bytes in 1284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:56.150584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:56.152089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:56.167594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8606e-52d0-477e-9231-d6791bca3e4d admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:56.233406 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714ee4e-cba9-48c3-bfab-e2b8a3f1de5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:56.234358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ee4e-cba9-48c3-bfab-e2b8a3f1de5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:56.234636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ee4e-cba9-48c3-bfab-e2b8a3f1de5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:56.234910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ee4e-cba9-48c3-bfab-e2b8a3f1de5d admin admin] PATH_INFO: `/v3/users/41fe5ea6287e4cc18659066f10e83e72` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:56.236297 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714ee4e-cba9-48c3-bfab-e2b8a3f1de5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=41fe5ea6287e4cc18659066f10e83e72)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:56.238281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714ee4e-cba9-48c3-bfab-e2b8a3f1de5d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:56.452810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f714ee4e-cba9-48c3-bfab-e2b8a3f1de5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41fe5ea6287e4cc18659066f10e83e72'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:56.453531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f714ee4e-cba9-48c3-bfab-e2b8a3f1de5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41fe5ea6287e4cc18659066f10e83e72'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:56.461958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f714ee4e-cba9-48c3-bfab-e2b8a3f1de5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41fe5ea6287e4cc18659066f10e83e72'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:56.479231 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f714ee4e-cba9-48c3-bfab-e2b8a3f1de5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41fe5ea6287e4cc18659066f10e83e72'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:56.594604 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-f714ee4e-cba9-48c3-bfab-e2b8a3f1de5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41fe5ea6287e4cc18659066f10e83e72'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:56.610378 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1205/2408] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:56 2022] DELETE /identity/v3/users/41fe5ea6287e4cc18659066f10e83e72 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:56.923319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:56.928718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:56.946154 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5096849-cac3-4a65-bd04-d992ee23b433 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:57.036017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:57.037418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:57.037945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:57.038428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] PATH_INFO: `/v3/projects/89810f71039342739650be12b7a8037d` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:57.039985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89810f71039342739650be12b7a8037d)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:57.042234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:57.238784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:57.271960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89810f71039342739650be12b7a8037d', 'request_id': 'req-1becf386-e67b-4843-b019-f291821047d9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:57.296369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89810f71039342739650be12b7a8037d', 'request_id': 'req-1becf386-e67b-4843-b019-f291821047d9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:57.306836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89810f71039342739650be12b7a8037d', 'request_id': 'req-1becf386-e67b-4843-b019-f291821047d9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:57.419818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] The token cache is being invalidate because project 89810f71039342739650be12b7a8037d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:57.420231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:57.421092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1becf386-e67b-4843-b019-f291821047d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:57.425982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1204/2409] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:56 2022] DELETE /identity/v3/projects/89810f71039342739650be12b7a8037d => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:57.428539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0be1c-c96c-4a70-955f-816a9f66cfd2 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:57.429472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0be1c-c96c-4a70-955f-816a9f66cfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:57.429981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0be1c-c96c-4a70-955f-816a9f66cfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:57.460936 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-7ed0be1c-c96c-4a70-955f-816a9f66cfd2 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:57.462142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7ed0be1c-c96c-4a70-955f-816a9f66cfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:57.522399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1206/2410] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:57.530347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb656fa-e923-4bb6-a793-74f199488f2e None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:57.530626 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb656fa-e923-4bb6-a793-74f199488f2e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:57.530900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb656fa-e923-4bb6-a793-74f199488f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:57.560307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-fbb656fa-e923-4bb6-a793-74f199488f2e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:57.561225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0be1c-c96c-4a70-955f-816a9f66cfd2 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:57.562342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-fbb656fa-e923-4bb6-a793-74f199488f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:57.562638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7ed0be1c-c96c-4a70-955f-816a9f66cfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:57.573160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1205/2411] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:57.579090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0be1c-c96c-4a70-955f-816a9f66cfd2 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:57.580766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb656fa-e923-4bb6-a793-74f199488f2e None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:57.583900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:57.584777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:57.585171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:57.585555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] PATH_INFO: `/v3/users/c94c65ae14e94796a3b3bd0bb29a91b9` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:57.586768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94c65ae14e94796a3b3bd0bb29a91b9)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:57.588773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:57.590696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb656fa-e923-4bb6-a793-74f199488f2e None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:57.595560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:57.596321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93442c-0071-4419-aa12-605236aa196a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:57.597040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93442c-0071-4419-aa12-605236aa196a admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:57.597392 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93442c-0071-4419-aa12-605236aa196a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:57.597859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93442c-0071-4419-aa12-605236aa196a admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:57.598699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93442c-0071-4419-aa12-605236aa196a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:23:57.599279 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93442c-0071-4419-aa12-605236aa196a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:57.603736 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93442c-0071-4419-aa12-605236aa196a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:57.617663 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1206/2412] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:23:57.621824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93442c-0071-4419-aa12-605236aa196a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:57.631677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93442c-0071-4419-aa12-605236aa196a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:57.636394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671eb1c-2d55-4fd1-bbc5-1944b89b3183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:57.637213 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671eb1c-2d55-4fd1-bbc5-1944b89b3183 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:57.647577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671eb1c-2d55-4fd1-bbc5-1944b89b3183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:57.647577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671eb1c-2d55-4fd1-bbc5-1944b89b3183 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:57.647577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671eb1c-2d55-4fd1-bbc5-1944b89b3183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:57.647577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671eb1c-2d55-4fd1-bbc5-1944b89b3183 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:57.762092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1207/2413] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:57 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:57.766939 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671eb1c-2d55-4fd1-bbc5-1944b89b3183 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:57.797156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94c65ae14e94796a3b3bd0bb29a91b9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:57.797977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94c65ae14e94796a3b3bd0bb29a91b9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:57.808548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94c65ae14e94796a3b3bd0bb29a91b9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:57.828326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94c65ae14e94796a3b3bd0bb29a91b9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:57.852973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671eb1c-2d55-4fd1-bbc5-1944b89b3183 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:57.857664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af916777-6ce0-4c0f-bf5d-77fcb22dff77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:57.858266 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af916777-6ce0-4c0f-bf5d-77fcb22dff77 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:57.858550 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af916777-6ce0-4c0f-bf5d-77fcb22dff77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:57.858842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af916777-6ce0-4c0f-bf5d-77fcb22dff77 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:57.859478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af916777-6ce0-4c0f-bf5d-77fcb22dff77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:57.861747 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af916777-6ce0-4c0f-bf5d-77fcb22dff77 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:57.914301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94c65ae14e94796a3b3bd0bb29a91b9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:57.920759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1207/2414] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:57 2022] DELETE /identity/v3/users/c94c65ae14e94796a3b3bd0bb29a91b9 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:58.035968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1208/2415] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:57 2022] POST /identity/v3/users => generated 288 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:58.038792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:58.040271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:58.110796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38287e-3fa5-4347-b860-ac7d9214e317 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:58.116234 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fbde-5aea-41d6-a14f-78ac282dd0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:58.116990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fbde-5aea-41d6-a14f-78ac282dd0af admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.117229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fbde-5aea-41d6-a14f-78ac282dd0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.117527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fbde-5aea-41d6-a14f-78ac282dd0af admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.118268 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7fbde-5aea-41d6-a14f-78ac282dd0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:58.120013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7fbde-5aea-41d6-a14f-78ac282dd0af admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:58.130516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1208/2416] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:58.135570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af916777-6ce0-4c0f-bf5d-77fcb22dff77 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:58.147206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af916777-6ce0-4c0f-bf5d-77fcb22dff77 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:58.151921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ee699-1060-4932-94e7-3b751d74c845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:58.152696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ee699-1060-4932-94e7-3b751d74c845 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.153016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ee699-1060-4932-94e7-3b751d74c845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.153288 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ee699-1060-4932-94e7-3b751d74c845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.154047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ee699-1060-4932-94e7-3b751d74c845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:58.155669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ee699-1060-4932-94e7-3b751d74c845 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:58.165946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1209/2417] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:58.170389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fbde-5aea-41d6-a14f-78ac282dd0af admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:58.181336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fbde-5aea-41d6-a14f-78ac282dd0af admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:58.186029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc730a61-4368-43cc-8adb-afca280846ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:58.186702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc730a61-4368-43cc-8adb-afca280846ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.187027 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc730a61-4368-43cc-8adb-afca280846ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.187305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc730a61-4368-43cc-8adb-afca280846ef admin admin] PATH_INFO: `/v3/projects/d021de128efb4a7b8ddc9b500edb0eb3/users/bb5859187ff945a99360ae6fd0cfd5b1/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.203170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc730a61-4368-43cc-8adb-afca280846ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d021de128efb4a7b8ddc9b500edb0eb3, user_id=bb5859187ff945a99360ae6fd0cfd5b1, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:58.205913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc730a61-4368-43cc-8adb-afca280846ef admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:58.224651 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ee699-1060-4932-94e7-3b751d74c845 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:58.226781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c65ee699-1060-4932-94e7-3b751d74c845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:58.255195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ee699-1060-4932-94e7-3b751d74c845 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:58.260017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:58.260885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.261208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.261699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] PATH_INFO: `/v3/projects/8824664cdd104c308141264985a547a1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.262784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8824664cdd104c308141264985a547a1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:58.265007 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:58.337551 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1209/2418] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:58 2022] PUT /identity/v3/projects/d021de128efb4a7b8ddc9b500edb0eb3/users/bb5859187ff945a99360ae6fd0cfd5b1/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:58.355247 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cd099-0529-452d-98ef-a883c7f857c3 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.355539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cd099-0529-452d-98ef-a883c7f857c3 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.355820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cd099-0529-452d-98ef-a883c7f857c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.384638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b95cd099-0529-452d-98ef-a883c7f857c3 None None] MFA Rules not processed for user `bb5859187ff945a99360ae6fd0cfd5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:58.385888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b95cd099-0529-452d-98ef-a883c7f857c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:58.457267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8824664cdd104c308141264985a547a1', 'request_id': 'req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:58.459423 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1210/2419] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:58.464531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cd099-0529-452d-98ef-a883c7f857c3 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:58.464728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8824664cdd104c308141264985a547a1', 'request_id': 'req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:58.472416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8824664cdd104c308141264985a547a1', 'request_id': 'req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:58.526594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cd099-0529-452d-98ef-a883c7f857c3 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:58.582001 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72962f2-293c-4ec9-be64-6fa5c678af67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:58.582523 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72962f2-293c-4ec9-be64-6fa5c678af67 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.582799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72962f2-293c-4ec9-be64-6fa5c678af67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.583071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72962f2-293c-4ec9-be64-6fa5c678af67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.583675 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72962f2-293c-4ec9-be64-6fa5c678af67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:58.585855 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72962f2-293c-4ec9-be64-6fa5c678af67 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:58.593824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] The token cache is being invalidate because project 8824664cdd104c308141264985a547a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:58.594117 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:58.598044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4170bef-e9f9-4ea7-80d1-a3deff3b65a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:58.598044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1210/2420] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:58 2022] DELETE /identity/v3/projects/8824664cdd104c308141264985a547a1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:58.602065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22fefe-5130-448c-8664-18c7ffe88191 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.602357 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22fefe-5130-448c-8664-18c7ffe88191 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.602632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22fefe-5130-448c-8664-18c7ffe88191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.622966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-cc22fefe-5130-448c-8664-18c7ffe88191 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:58.624169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-cc22fefe-5130-448c-8664-18c7ffe88191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:58.632799 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1211/2421] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:58.640491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac23a54-14f3-472b-954f-d6a86a4c52cd None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.640765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac23a54-14f3-472b-954f-d6a86a4c52cd None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.641078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac23a54-14f3-472b-954f-d6a86a4c52cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.660718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6ac23a54-14f3-472b-954f-d6a86a4c52cd None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:58.661932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6ac23a54-14f3-472b-954f-d6a86a4c52cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:58.670819 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1212/2422] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:58.674897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac23a54-14f3-472b-954f-d6a86a4c52cd None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:58.684017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac23a54-14f3-472b-954f-d6a86a4c52cd None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:58.688595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d03d7d-f6dc-49d0-9850-bd62ef66ded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:58.689200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d03d7d-f6dc-49d0-9850-bd62ef66ded4 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.689483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d03d7d-f6dc-49d0-9850-bd62ef66ded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.689756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d03d7d-f6dc-49d0-9850-bd62ef66ded4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.690415 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d03d7d-f6dc-49d0-9850-bd62ef66ded4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:23:58.690720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d03d7d-f6dc-49d0-9850-bd62ef66ded4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:58.692307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d03d7d-f6dc-49d0-9850-bd62ef66ded4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:58.703881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1213/2423] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:23:58.708587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d03d7d-f6dc-49d0-9850-bd62ef66ded4 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:58.718958 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d03d7d-f6dc-49d0-9850-bd62ef66ded4 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:58.726295 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1211/2424] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:58 2022] POST /identity/v3/projects => generated 377 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:58.726473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad5f8f-49be-49db-a32c-8929fcec5635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:58.726473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad5f8f-49be-49db-a32c-8929fcec5635 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.726473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad5f8f-49be-49db-a32c-8929fcec5635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.726473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad5f8f-49be-49db-a32c-8929fcec5635 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.726781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad5f8f-49be-49db-a32c-8929fcec5635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:58.727553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72962f2-293c-4ec9-be64-6fa5c678af67 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:58.731601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad5f8f-49be-49db-a32c-8929fcec5635 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:58.742747 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e72962f2-293c-4ec9-be64-6fa5c678af67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:58.789023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=10,keystone:SELECT=85,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:58.814148 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72962f2-293c-4ec9-be64-6fa5c678af67 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:58.820837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245cd11-b612-4f3f-b43c-3fd64ef66491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:58.821581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245cd11-b612-4f3f-b43c-3fd64ef66491 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.821881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245cd11-b612-4f3f-b43c-3fd64ef66491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.822168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245cd11-b612-4f3f-b43c-3fd64ef66491 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.822875 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245cd11-b612-4f3f-b43c-3fd64ef66491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:58.825464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1214/2425] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:58 2022] POST /identity/v3/projects => generated 401 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:58.825687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245cd11-b612-4f3f-b43c-3fd64ef66491 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:58.829712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad5f8f-49be-49db-a32c-8929fcec5635 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:58.902472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad5f8f-49be-49db-a32c-8929fcec5635 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:58.909565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b15bd-59be-47d9-be6e-7ca78cf212a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:58.911115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b15bd-59be-47d9-be6e-7ca78cf212a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.911402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b15bd-59be-47d9-be6e-7ca78cf212a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.911677 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b15bd-59be-47d9-be6e-7ca78cf212a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.912795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b15bd-59be-47d9-be6e-7ca78cf212a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:58.915011 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b15bd-59be-47d9-be6e-7ca78cf212a2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:58.942777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1212/2426] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:58 2022] POST /identity/v3/users => generated 295 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:23:58.944522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245cd11-b612-4f3f-b43c-3fd64ef66491 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:58.956232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245cd11-b612-4f3f-b43c-3fd64ef66491 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:58.964591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020edd-240e-422b-abd0-d3259c9b9e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:58.965449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020edd-240e-422b-abd0-d3259c9b9e78 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:58.966518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020edd-240e-422b-abd0-d3259c9b9e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:58.966518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020edd-240e-422b-abd0-d3259c9b9e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:58.967003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67020edd-240e-422b-abd0-d3259c9b9e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:58.968755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67020edd-240e-422b-abd0-d3259c9b9e78 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:58.984214 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1213/2427] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:58.988710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020edd-240e-422b-abd0-d3259c9b9e78 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.000093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020edd-240e-422b-abd0-d3259c9b9e78 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.006238 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74223c0c-3d07-4ba7-ac39-ebd95db0b2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.007783 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74223c0c-3d07-4ba7-ac39-ebd95db0b2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.008073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74223c0c-3d07-4ba7-ac39-ebd95db0b2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.008417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74223c0c-3d07-4ba7-ac39-ebd95db0b2bf admin admin] PATH_INFO: `/v3/projects/b88c4f600bf74d46a0cf6e655ead0658/users/72781b5ab4674782b18f1d78576fb6b1/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.030179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74223c0c-3d07-4ba7-ac39-ebd95db0b2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b88c4f600bf74d46a0cf6e655ead0658, user_id=72781b5ab4674782b18f1d78576fb6b1, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.032904 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74223c0c-3d07-4ba7-ac39-ebd95db0b2bf admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:59.063710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1215/2428] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:58 2022] POST /identity/v3/users => generated 301 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:59.067668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b15bd-59be-47d9-be6e-7ca78cf212a2 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.099296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b15bd-59be-47d9-be6e-7ca78cf212a2 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.099296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387256e5-9b70-4e12-8783-2d679c56ab81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.099296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387256e5-9b70-4e12-8783-2d679c56ab81 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.099296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387256e5-9b70-4e12-8783-2d679c56ab81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.099707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387256e5-9b70-4e12-8783-2d679c56ab81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.099707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387256e5-9b70-4e12-8783-2d679c56ab81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.099707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387256e5-9b70-4e12-8783-2d679c56ab81 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:59.114365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1216/2429] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:59.117696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387256e5-9b70-4e12-8783-2d679c56ab81 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.128479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387256e5-9b70-4e12-8783-2d679c56ab81 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.133657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eb517-c224-4a96-9e9b-722513aa3c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.134253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb517-c224-4a96-9e9b-722513aa3c5d admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.134533 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb517-c224-4a96-9e9b-722513aa3c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.138162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb517-c224-4a96-9e9b-722513aa3c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.138162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eb517-c224-4a96-9e9b-722513aa3c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.138162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eb517-c224-4a96-9e9b-722513aa3c5d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:59.149883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1217/2430] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:23:59.150755 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eb517-c224-4a96-9e9b-722513aa3c5d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.152348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ba8eb517-c224-4a96-9e9b-722513aa3c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:59.169988 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1214/2431] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:58 2022] PUT /identity/v3/projects/b88c4f600bf74d46a0cf6e655ead0658/users/72781b5ab4674782b18f1d78576fb6b1/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:59.170906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74223c0c-3d07-4ba7-ac39-ebd95db0b2bf admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.213110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eb517-c224-4a96-9e9b-722513aa3c5d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.239593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74223c0c-3d07-4ba7-ac39-ebd95db0b2bf admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.245221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159dc38-91bd-4513-923a-b764a34ad112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.247287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159dc38-91bd-4513-923a-b764a34ad112 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.247287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159dc38-91bd-4513-923a-b764a34ad112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.247287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159dc38-91bd-4513-923a-b764a34ad112 admin admin] PATH_INFO: `/v3/projects/ec5fe346cd4e4751b5e407cc67d37f2c/users/2dbe7a9833e64b6b8fcad4cc1298d4e6/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.260850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159dc38-91bd-4513-923a-b764a34ad112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec5fe346cd4e4751b5e407cc67d37f2c, user_id=2dbe7a9833e64b6b8fcad4cc1298d4e6, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.264903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de58a77-e75c-49e8-9d2c-69db134f7653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.265064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159dc38-91bd-4513-923a-b764a34ad112 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:59.265150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de58a77-e75c-49e8-9d2c-69db134f7653 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.265150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de58a77-e75c-49e8-9d2c-69db134f7653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.265150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de58a77-e75c-49e8-9d2c-69db134f7653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.268538 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0de58a77-e75c-49e8-9d2c-69db134f7653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:59.277381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de58a77-e75c-49e8-9d2c-69db134f7653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.278903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de58a77-e75c-49e8-9d2c-69db134f7653 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:59.288917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1218/2432] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:59.289616 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de58a77-e75c-49e8-9d2c-69db134f7653 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.305638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de58a77-e75c-49e8-9d2c-69db134f7653 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.310788 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ec5d5-ce9f-496a-be7d-7e29e399389e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.311340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ec5d5-ce9f-496a-be7d-7e29e399389e admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.311757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ec5d5-ce9f-496a-be7d-7e29e399389e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.312045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ec5d5-ce9f-496a-be7d-7e29e399389e admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.312690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ec5d5-ce9f-496a-be7d-7e29e399389e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.314325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ec5d5-ce9f-496a-be7d-7e29e399389e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:59.325964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1219/2433] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:23:59.330237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ec5d5-ce9f-496a-be7d-7e29e399389e admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.340123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ec5d5-ce9f-496a-be7d-7e29e399389e admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.345451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32808105-01dc-4d2b-a110-927cc30354a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.346161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32808105-01dc-4d2b-a110-927cc30354a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.346451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32808105-01dc-4d2b-a110-927cc30354a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.346756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32808105-01dc-4d2b-a110-927cc30354a9 admin admin] PATH_INFO: `/v3/projects/b88c4f600bf74d46a0cf6e655ead0658/users/72781b5ab4674782b18f1d78576fb6b1/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.348718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32808105-01dc-4d2b-a110-927cc30354a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b88c4f600bf74d46a0cf6e655ead0658, user_id=72781b5ab4674782b18f1d78576fb6b1, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.352226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32808105-01dc-4d2b-a110-927cc30354a9 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:59.362842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1220/2434] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:59 2022] PUT /identity/v3/projects/b88c4f600bf74d46a0cf6e655ead0658/users/72781b5ab4674782b18f1d78576fb6b1/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:59.374077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8ad46-a6ff-48d6-8a5b-c52e495087a7 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.374305 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8ad46-a6ff-48d6-8a5b-c52e495087a7 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.374580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8ad46-a6ff-48d6-8a5b-c52e495087a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.402602 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-59c8ad46-a6ff-48d6-8a5b-c52e495087a7 None None] MFA Rules not processed for user `72781b5ab4674782b18f1d78576fb6b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:59.403785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-59c8ad46-a6ff-48d6-8a5b-c52e495087a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:59.434126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1215/2435] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:23:59 2022] PUT /identity/v3/projects/ec5fe346cd4e4751b5e407cc67d37f2c/users/2dbe7a9833e64b6b8fcad4cc1298d4e6/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:59.446575 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59895291-31fe-4b8c-8069-2586c2f3c89a None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.446868 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59895291-31fe-4b8c-8069-2586c2f3c89a None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.447149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59895291-31fe-4b8c-8069-2586c2f3c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.477950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1221/2436] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:59.478408 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-59895291-31fe-4b8c-8069-2586c2f3c89a None None] MFA Rules not processed for user `2dbe7a9833e64b6b8fcad4cc1298d4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:59.479803 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-59895291-31fe-4b8c-8069-2586c2f3c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:59.489467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8ad46-a6ff-48d6-8a5b-c52e495087a7 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.491021 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-59c8ad46-a6ff-48d6-8a5b-c52e495087a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:59.551687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8ad46-a6ff-48d6-8a5b-c52e495087a7 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.552763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1216/2437] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:23:59 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:23:59.565050 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59895291-31fe-4b8c-8069-2586c2f3c89a None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.615140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9d0ef-917b-4a9c-8335-c18297d08b5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.615687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9d0ef-917b-4a9c-8335-c18297d08b5f service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.615954 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9d0ef-917b-4a9c-8335-c18297d08b5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.616337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9d0ef-917b-4a9c-8335-c18297d08b5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.618167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-99d9d0ef-917b-4a9c-8335-c18297d08b5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:59.637473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59895291-31fe-4b8c-8069-2586c2f3c89a None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.642166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ac596-469c-4631-9ba4-b80ba6eb167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.642776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ac596-469c-4631-9ba4-b80ba6eb167c admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.643067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ac596-469c-4631-9ba4-b80ba6eb167c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.643377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ac596-469c-4631-9ba4-b80ba6eb167c admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.644015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ac596-469c-4631-9ba4-b80ba6eb167c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.645978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ac596-469c-4631-9ba4-b80ba6eb167c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:59.681531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9d0ef-917b-4a9c-8335-c18297d08b5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.683000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9d0ef-917b-4a9c-8335-c18297d08b5f service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:59.692284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1222/2438] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:23:59.693178 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9d0ef-917b-4a9c-8335-c18297d08b5f service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.695457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-99d9d0ef-917b-4a9c-8335-c18297d08b5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:59.708887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9d0ef-917b-4a9c-8335-c18297d08b5f service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.713867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.714492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.714776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.715056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] PATH_INFO: `/v3/users/a2f6a1da30a64131a41e5d5fa3883d5d` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.716263 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f6a1da30a64131a41e5d5fa3883d5d)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.718267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:59.772798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1217/2439] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:23:59 2022] POST /identity/v3/projects => generated 401 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:23:59.776799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ac596-469c-4631-9ba4-b80ba6eb167c admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.815098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:59.860536 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ac596-469c-4631-9ba4-b80ba6eb167c admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.865443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce282b8-7fec-4596-90c7-df04a596e83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.866785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce282b8-7fec-4596-90c7-df04a596e83e admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.867076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce282b8-7fec-4596-90c7-df04a596e83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.867388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce282b8-7fec-4596-90c7-df04a596e83e admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.868078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce282b8-7fec-4596-90c7-df04a596e83e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.870028 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce282b8-7fec-4596-90c7-df04a596e83e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:00.003338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1218/2440] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:23:59 2022] POST /identity/v3/users => generated 307 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:24:00.003483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f6a1da30a64131a41e5d5fa3883d5d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:00.003483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f6a1da30a64131a41e5d5fa3883d5d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:00.003655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ee2e9-8d07-4324-9d67-4995f93b5078 None None] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.003655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ee2e9-8d07-4324-9d67-4995f93b5078 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.003655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ee2e9-8d07-4324-9d67-4995f93b5078 None None] PATH_INFO: `/` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.003655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1219/2441] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:24:00.003655 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ee2e9-8d07-4324-9d67-4995f93b5078 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:00.007408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f6a1da30a64131a41e5d5fa3883d5d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:00.008006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f6a1da30a64131a41e5d5fa3883d5d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:00.015681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ee2e9-8d07-4324-9d67-4995f93b5078 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:00.020273 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348cf5c-7c88-44bc-a532-f4cefb6de091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:00.020884 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348cf5c-7c88-44bc-a532-f4cefb6de091 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.021172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348cf5c-7c88-44bc-a532-f4cefb6de091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.021451 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348cf5c-7c88-44bc-a532-f4cefb6de091 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.022089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348cf5c-7c88-44bc-a532-f4cefb6de091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:00.023686 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348cf5c-7c88-44bc-a532-f4cefb6de091 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:00.033888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1220/2442] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:00.035076 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77f25f-dbd8-4366-bf45-d0e9e41883e4 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.035364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77f25f-dbd8-4366-bf45-d0e9e41883e4 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.035640 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77f25f-dbd8-4366-bf45-d0e9e41883e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.056089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-da77f25f-dbd8-4366-bf45-d0e9e41883e4 None None] MFA Rules not processed for user `ae9b6525009b41288dd918437acee92a`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:00.057685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-da77f25f-dbd8-4366-bf45-d0e9e41883e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:00.117997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f6a1da30a64131a41e5d5fa3883d5d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:00.121763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1221/2443] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:24:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:00.126808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77f25f-dbd8-4366-bf45-d0e9e41883e4 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:00.126808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-da77f25f-dbd8-4366-bf45-d0e9e41883e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:00.139710 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77f25f-dbd8-4366-bf45-d0e9e41883e4 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:00.145055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1b6b0-df39-4d22-9c5b-71096171d356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:00.145901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1b6b0-df39-4d22-9c5b-71096171d356 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.146242 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1b6b0-df39-4d22-9c5b-71096171d356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.146695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1b6b0-df39-4d22-9c5b-71096171d356 admin admin] PATH_INFO: `/v3/projects/4373e2095dfe450c8a05a813dd409c2f/users/cf30ff2f64a34489a86bc1ec2e88e7b2/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.146953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1223/2444] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:59 2022] DELETE /identity/v3/users/a2f6a1da30a64131a41e5d5fa3883d5d => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:00.151718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:00.161204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:00.192664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1b6b0-df39-4d22-9c5b-71096171d356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4373e2095dfe450c8a05a813dd409c2f, user_id=cf30ff2f64a34489a86bc1ec2e88e7b2, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:00.192664 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1b6b0-df39-4d22-9c5b-71096171d356 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:00.229308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfc777-5b21-49fb-a770-898b85ed42f7 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:00.233434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cdd9b-7ccf-4632-93ad-feda91049a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:00.234150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cdd9b-7ccf-4632-93ad-feda91049a78 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.234526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cdd9b-7ccf-4632-93ad-feda91049a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.234893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cdd9b-7ccf-4632-93ad-feda91049a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.248067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cdd9b-7ccf-4632-93ad-feda91049a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:00.248067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cdd9b-7ccf-4632-93ad-feda91049a78 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:00.255546 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1224/2445] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:00.256770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13f33-83f4-4d46-829a-ae77bd1f2831 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.257324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13f33-83f4-4d46-829a-ae77bd1f2831 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.257666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13f33-83f4-4d46-829a-ae77bd1f2831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.276325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1222/2446] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:00 2022] PUT /identity/v3/projects/4373e2095dfe450c8a05a813dd409c2f/users/cf30ff2f64a34489a86bc1ec2e88e7b2/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:00.281484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1b6b0-df39-4d22-9c5b-71096171d356 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:00.309916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-70c13f33-83f4-4d46-829a-ae77bd1f2831 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:00.309916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-70c13f33-83f4-4d46-829a-ae77bd1f2831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:00.401192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1225/2447] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:00.401192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d847ab-ccc7-4186-ace7-0de1ddecd9d6 None None] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.401192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d847ab-ccc7-4186-ace7-0de1ddecd9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.401192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d847ab-ccc7-4186-ace7-0de1ddecd9d6 None None] PATH_INFO: `/` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.401192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1226/2448] 10.222.0.30 () {54 vars in 921 bytes} [Wed Aug 24 10:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:24:00.402971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec4dd2-bd98-4d3e-84f7-596a0a6c5656 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.403395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec4dd2-bd98-4d3e-84f7-596a0a6c5656 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.403748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec4dd2-bd98-4d3e-84f7-596a0a6c5656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.421365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1b6b0-df39-4d22-9c5b-71096171d356 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:00.426318 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6dec4dd2-bd98-4d3e-84f7-596a0a6c5656 None None] MFA Rules not processed for user `7cd6d2291f934fbdb25c410c2c3abcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:00.426453 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d6ede-c80e-470d-83ca-043a999f4ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:00.427150 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6ede-c80e-470d-83ca-043a999f4ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.427497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6ede-c80e-470d-83ca-043a999f4ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.427839 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6ede-c80e-470d-83ca-043a999f4ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.429229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d6ede-c80e-470d-83ca-043a999f4ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:00.429450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6dec4dd2-bd98-4d3e-84f7-596a0a6c5656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:00.431111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d6ede-c80e-470d-83ca-043a999f4ffc admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:00.437434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:00.443756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1223/2449] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:24:00.445304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d6ede-c80e-470d-83ca-043a999f4ffc admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:00.507923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d6ede-c80e-470d-83ca-043a999f4ffc admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:00.509257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1227/2450] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Aug 24 10:24:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:00.509823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec4dd2-bd98-4d3e-84f7-596a0a6c5656 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:00.512317 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589140f2-e473-42db-b813-6a7fe0f084c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:00.512992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589140f2-e473-42db-b813-6a7fe0f084c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.513270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589140f2-e473-42db-b813-6a7fe0f084c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.513540 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589140f2-e473-42db-b813-6a7fe0f084c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.525002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589140f2-e473-42db-b813-6a7fe0f084c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:00.526167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec4dd2-bd98-4d3e-84f7-596a0a6c5656 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:00.526686 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589140f2-e473-42db-b813-6a7fe0f084c5 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:00.534215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298d9f6-2d0b-4321-bf44-1801dc782e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:00.534900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298d9f6-2d0b-4321-bf44-1801dc782e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.535186 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298d9f6-2d0b-4321-bf44-1801dc782e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.535465 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298d9f6-2d0b-4321-bf44-1801dc782e1c admin admin] PATH_INFO: `/v3/projects/4373e2095dfe450c8a05a813dd409c2f/users/cf30ff2f64a34489a86bc1ec2e88e7b2/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.538789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1224/2451] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:00.539709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589140f2-e473-42db-b813-6a7fe0f084c5 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:00.539837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298d9f6-2d0b-4321-bf44-1801dc782e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4373e2095dfe450c8a05a813dd409c2f, user_id=cf30ff2f64a34489a86bc1ec2e88e7b2, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:00.544565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298d9f6-2d0b-4321-bf44-1801dc782e1c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:00.549052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589140f2-e473-42db-b813-6a7fe0f084c5 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:00.553939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da42b23-bb66-4078-86fa-0f04f239ba1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:00.554728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da42b23-bb66-4078-86fa-0f04f239ba1f service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.555692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da42b23-bb66-4078-86fa-0f04f239ba1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.555692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da42b23-bb66-4078-86fa-0f04f239ba1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.561490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1228/2452] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:00 2022] PUT /identity/v3/projects/4373e2095dfe450c8a05a813dd409c2f/users/cf30ff2f64a34489a86bc1ec2e88e7b2/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:00.574532 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14465f9a-95a4-4376-a01e-79cb78bf10aa None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.574990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14465f9a-95a4-4376-a01e-79cb78bf10aa None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.575170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da42b23-bb66-4078-86fa-0f04f239ba1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:00.575526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14465f9a-95a4-4376-a01e-79cb78bf10aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.576787 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da42b23-bb66-4078-86fa-0f04f239ba1f service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:00.594072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:00.598689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1225/2453] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:00.611324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-14465f9a-95a4-4376-a01e-79cb78bf10aa None None] MFA Rules not processed for user `cf30ff2f64a34489a86bc1ec2e88e7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:00.612538 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-14465f9a-95a4-4376-a01e-79cb78bf10aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:00.693109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1229/2454] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:00 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:00.705144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da42b23-bb66-4078-86fa-0f04f239ba1f service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:00.706746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-9da42b23-bb66-4078-86fa-0f04f239ba1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:00.773059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da42b23-bb66-4078-86fa-0f04f239ba1f service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:00.831497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0c7e2-7898-4baf-839a-f75b5b7ca2fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:00.832330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0c7e2-7898-4baf-839a-f75b5b7ca2fe service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:00.832571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0c7e2-7898-4baf-839a-f75b5b7ca2fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:00.833078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0c7e2-7898-4baf-839a-f75b5b7ca2fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:00.844157 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0c7e2-7898-4baf-839a-f75b5b7ca2fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:00.845670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0c7e2-7898-4baf-839a-f75b5b7ca2fe service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:00.856208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1226/2455] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:00 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:01.017382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:01.141143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14465f9a-95a4-4376-a01e-79cb78bf10aa None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:01.142992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-14465f9a-95a4-4376-a01e-79cb78bf10aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:01.153985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0c7e2-7898-4baf-839a-f75b5b7ca2fe service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:01.165020 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0c7e2-7898-4baf-839a-f75b5b7ca2fe service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:01.177003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae022e15-ab2f-4d3f-834d-80e9497e9f2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:01.177199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae022e15-ab2f-4d3f-834d-80e9497e9f2f service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:01.177459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae022e15-ab2f-4d3f-834d-80e9497e9f2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:01.177737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae022e15-ab2f-4d3f-834d-80e9497e9f2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:01.182508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ae022e15-ab2f-4d3f-834d-80e9497e9f2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:01.264176 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae022e15-ab2f-4d3f-834d-80e9497e9f2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:01.266410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae022e15-ab2f-4d3f-834d-80e9497e9f2f service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:01.270633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14465f9a-95a4-4376-a01e-79cb78bf10aa None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:01.274455 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:01.275090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:01.275369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:01.275739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] PATH_INFO: `/v3/projects/712466a083bd40d89bc3866c13ad4930` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:01.275973 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1227/2456] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:01 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:01.276982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=712466a083bd40d89bc3866c13ad4930)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:01.279045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:01.291452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:01.443228 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '712466a083bd40d89bc3866c13ad4930', 'request_id': 'req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:01.450784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '712466a083bd40d89bc3866c13ad4930', 'request_id': 'req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:01.459393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '712466a083bd40d89bc3866c13ad4930', 'request_id': 'req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:01.543598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] The token cache is being invalidate because project 712466a083bd40d89bc3866c13ad4930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:01.546215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:01.546215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:01.548562 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1230/2457] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:01 2022] DELETE /identity/v3/projects/712466a083bd40d89bc3866c13ad4930 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:01.552773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae022e15-ab2f-4d3f-834d-80e9497e9f2f service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:01.565595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae022e15-ab2f-4d3f-834d-80e9497e9f2f service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:01.626396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a399ee-8059-4c77-a003-9da3474eebea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:01.627023 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a399ee-8059-4c77-a003-9da3474eebea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:01.627404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a399ee-8059-4c77-a003-9da3474eebea admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:01.627679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a399ee-8059-4c77-a003-9da3474eebea admin admin] PATH_INFO: `/v3/users/461ef58840464370bef92620d335615e` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:01.628830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a399ee-8059-4c77-a003-9da3474eebea admin admin] RBAC: Authorizing `identity:delete_user(user_id=461ef58840464370bef92620d335615e)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:01.630811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a399ee-8059-4c77-a003-9da3474eebea admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:01.876839 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a5a399ee-8059-4c77-a003-9da3474eebea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '461ef58840464370bef92620d335615e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:01.877545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a5a399ee-8059-4c77-a003-9da3474eebea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '461ef58840464370bef92620d335615e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:01.885255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a5a399ee-8059-4c77-a003-9da3474eebea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '461ef58840464370bef92620d335615e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:01.899354 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a5a399ee-8059-4c77-a003-9da3474eebea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '461ef58840464370bef92620d335615e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:02.007775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:02.101289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-a5a399ee-8059-4c77-a003-9da3474eebea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '461ef58840464370bef92620d335615e'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:02.108177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1228/2458] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:01 2022] DELETE /identity/v3/users/461ef58840464370bef92620d335615e => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:02.360307 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:03.007683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:03.010420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:03.023385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7b71a-bed7-4954-a42f-d8723ca9cdcb admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:03.082729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:03.083349 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:03.083578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:03.083855 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] PATH_INFO: `/v3/projects/c45a7c273e3f43d7a2eefe342074cff8` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:03.085654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c45a7c273e3f43d7a2eefe342074cff8)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:03.087573 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:03.195867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c45a7c273e3f43d7a2eefe342074cff8', 'request_id': 'req-f97f296e-99ca-43a1-a6cc-e094c0898a49'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:03.202449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c45a7c273e3f43d7a2eefe342074cff8', 'request_id': 'req-f97f296e-99ca-43a1-a6cc-e094c0898a49'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:03.209883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c45a7c273e3f43d7a2eefe342074cff8', 'request_id': 'req-f97f296e-99ca-43a1-a6cc-e094c0898a49'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:03.308480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] The token cache is being invalidate because project c45a7c273e3f43d7a2eefe342074cff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:03.308771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:03.309458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f97f296e-99ca-43a1-a6cc-e094c0898a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:03.311182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1231/2459] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:03 2022] DELETE /identity/v3/projects/c45a7c273e3f43d7a2eefe342074cff8 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:03.316406 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ff2ee-593f-4c8b-b80a-621dd24bf9a2 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:03.317003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ff2ee-593f-4c8b-b80a-621dd24bf9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:03.317003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ff2ee-593f-4c8b-b80a-621dd24bf9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:03.342455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b71ff2ee-593f-4c8b-b80a-621dd24bf9a2 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:03.343980 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b71ff2ee-593f-4c8b-b80a-621dd24bf9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:03.413885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1229/2460] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:03.479394 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ded8d-fb8f-44a2-b37b-97b76ce24045 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:03.479676 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ded8d-fb8f-44a2-b37b-97b76ce24045 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:03.479948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ded8d-fb8f-44a2-b37b-97b76ce24045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:03.501182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-5c0ded8d-fb8f-44a2-b37b-97b76ce24045 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:03.502431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5c0ded8d-fb8f-44a2-b37b-97b76ce24045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:03.510595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1232/2461] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:24:03.515032 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ff2ee-593f-4c8b-b80a-621dd24bf9a2 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:03.525498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ff2ee-593f-4c8b-b80a-621dd24bf9a2 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:03.530507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766989d-154e-43f6-8541-caa15ce92aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:03.531183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766989d-154e-43f6-8541-caa15ce92aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:03.531562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766989d-154e-43f6-8541-caa15ce92aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:03.531903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766989d-154e-43f6-8541-caa15ce92aa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:03.532628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766989d-154e-43f6-8541-caa15ce92aa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:24:03.533036 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766989d-154e-43f6-8541-caa15ce92aa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:03.534808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766989d-154e-43f6-8541-caa15ce92aa5 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:03.552905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1230/2462] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:24:03.556298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ded8d-fb8f-44a2-b37b-97b76ce24045 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:03.566739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ded8d-fb8f-44a2-b37b-97b76ce24045 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:03.571482 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34e26a-dbb8-49b9-8fbf-1f270b273be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:03.572085 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e26a-dbb8-49b9-8fbf-1f270b273be2 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:03.572368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e26a-dbb8-49b9-8fbf-1f270b273be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:03.572640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e26a-dbb8-49b9-8fbf-1f270b273be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:03.573297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34e26a-dbb8-49b9-8fbf-1f270b273be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:03.575418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34e26a-dbb8-49b9-8fbf-1f270b273be2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:03.667256 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1233/2463] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:24:03 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:03.668424 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766989d-154e-43f6-8541-caa15ce92aa5 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:03.676879 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:03.681560 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766989d-154e-43f6-8541-caa15ce92aa5 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:03.740413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237e41e-f7b4-4f81-830e-0beca8a09995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:03.741656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237e41e-f7b4-4f81-830e-0beca8a09995 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:03.741940 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237e41e-f7b4-4f81-830e-0beca8a09995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:03.742358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237e41e-f7b4-4f81-830e-0beca8a09995 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:03.742994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237e41e-f7b4-4f81-830e-0beca8a09995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:03.744925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237e41e-f7b4-4f81-830e-0beca8a09995 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:03.753887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:03.868019 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1231/2464] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:24:03 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:24:03.873386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34e26a-dbb8-49b9-8fbf-1f270b273be2 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:03.883839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34e26a-dbb8-49b9-8fbf-1f270b273be2 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:03.888324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891ef57-aefb-4aae-80b7-c0cdc695c610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:03.888942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891ef57-aefb-4aae-80b7-c0cdc695c610 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:03.893257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891ef57-aefb-4aae-80b7-c0cdc695c610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:03.893257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891ef57-aefb-4aae-80b7-c0cdc695c610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:03.893257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891ef57-aefb-4aae-80b7-c0cdc695c610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:03.893257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891ef57-aefb-4aae-80b7-c0cdc695c610 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:03.902868 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1234/2465] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:03.906895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237e41e-f7b4-4f81-830e-0beca8a09995 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:03.920328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237e41e-f7b4-4f81-830e-0beca8a09995 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:03.925344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac185c98-99b9-4647-915b-0e8f72c00e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:03.926025 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac185c98-99b9-4647-915b-0e8f72c00e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:03.926313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac185c98-99b9-4647-915b-0e8f72c00e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:03.926596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac185c98-99b9-4647-915b-0e8f72c00e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:03.927276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac185c98-99b9-4647-915b-0e8f72c00e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:03.928989 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac185c98-99b9-4647-915b-0e8f72c00e4b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:03.943986 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1232/2466] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:03.945970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891ef57-aefb-4aae-80b7-c0cdc695c610 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:03.956489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891ef57-aefb-4aae-80b7-c0cdc695c610 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:03.962333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5e3c5-1536-4aac-906b-ec2f4f54aa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:03.963004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5e3c5-1536-4aac-906b-ec2f4f54aa6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:03.963289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5e3c5-1536-4aac-906b-ec2f4f54aa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:03.963566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5e3c5-1536-4aac-906b-ec2f4f54aa6c admin admin] PATH_INFO: `/v3/projects/0d66e5f6205f4c6bb8dc4e50a94871a3/users/76181228273f4bbd85a5aabb8ef555a5/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.029197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:04.042041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e5e3c5-1536-4aac-906b-ec2f4f54aa6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d66e5f6205f4c6bb8dc4e50a94871a3, user_id=76181228273f4bbd85a5aabb8ef555a5, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:04.044678 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e5e3c5-1536-4aac-906b-ec2f4f54aa6c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:04.149593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1235/2467] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:03 2022] PUT /identity/v3/projects/0d66e5f6205f4c6bb8dc4e50a94871a3/users/76181228273f4bbd85a5aabb8ef555a5/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:04.160351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c92af3-d2f6-44c8-88a3-09b64c244713 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:04.160636 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c92af3-d2f6-44c8-88a3-09b64c244713 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:04.160979 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c92af3-d2f6-44c8-88a3-09b64c244713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.188820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-46c92af3-d2f6-44c8-88a3-09b64c244713 None None] MFA Rules not processed for user `76181228273f4bbd85a5aabb8ef555a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:04.190168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-46c92af3-d2f6-44c8-88a3-09b64c244713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:04.253457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1233/2468] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:04.256408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:04.258640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5e3c5-1536-4aac-906b-ec2f4f54aa6c admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:04.322411 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5e3c5-1536-4aac-906b-ec2f4f54aa6c admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:04.327066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c834560-457f-43e1-8f15-69cbd717a6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:04.327686 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c834560-457f-43e1-8f15-69cbd717a6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:04.327962 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c834560-457f-43e1-8f15-69cbd717a6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:04.328231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c834560-457f-43e1-8f15-69cbd717a6b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.328934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c834560-457f-43e1-8f15-69cbd717a6b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:04.330751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c834560-457f-43e1-8f15-69cbd717a6b5 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:04.418649 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1236/2469] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:24:04 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:04.424800 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c92af3-d2f6-44c8-88a3-09b64c244713 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:04.442193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c92af3-d2f6-44c8-88a3-09b64c244713 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:04.487782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:04.516424 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a69cf3-a3af-42ac-b559-fc9ddb6b9bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:04.517103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a69cf3-a3af-42ac-b559-fc9ddb6b9bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:04.517391 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a69cf3-a3af-42ac-b559-fc9ddb6b9bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:04.517735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a69cf3-a3af-42ac-b559-fc9ddb6b9bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.518552 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a69cf3-a3af-42ac-b559-fc9ddb6b9bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:04.520775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a69cf3-a3af-42ac-b559-fc9ddb6b9bd3 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:04.645976 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1234/2470] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:24:04 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:04.650374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c834560-457f-43e1-8f15-69cbd717a6b5 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:04.660664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c834560-457f-43e1-8f15-69cbd717a6b5 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:04.665424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f64da-a03f-41ed-bf99-0522f04959ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:04.665937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f64da-a03f-41ed-bf99-0522f04959ec admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:04.666221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f64da-a03f-41ed-bf99-0522f04959ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:04.666494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f64da-a03f-41ed-bf99-0522f04959ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.667127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f64da-a03f-41ed-bf99-0522f04959ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:04.668717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f64da-a03f-41ed-bf99-0522f04959ec admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:04.679926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1237/2471] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:24:04.684083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a69cf3-a3af-42ac-b559-fc9ddb6b9bd3 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:04.695073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a69cf3-a3af-42ac-b559-fc9ddb6b9bd3 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:04.700240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95b422-f9a5-4f28-b4d8-a08d143ee5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:04.700842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95b422-f9a5-4f28-b4d8-a08d143ee5ae admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:04.701162 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95b422-f9a5-4f28-b4d8-a08d143ee5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:04.701438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95b422-f9a5-4f28-b4d8-a08d143ee5ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.702085 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95b422-f9a5-4f28-b4d8-a08d143ee5ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:04.703685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95b422-f9a5-4f28-b4d8-a08d143ee5ae admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:04.718570 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1235/2472] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:04.719536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f64da-a03f-41ed-bf99-0522f04959ec admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:04.730000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f64da-a03f-41ed-bf99-0522f04959ec admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:04.734608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3d2d1-b030-47d5-95b8-88b2e874a281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:04.735544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3d2d1-b030-47d5-95b8-88b2e874a281 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:04.735824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3d2d1-b030-47d5-95b8-88b2e874a281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:04.736098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3d2d1-b030-47d5-95b8-88b2e874a281 admin admin] PATH_INFO: `/v3/projects/554393e62a6c4dacb08201cdf828b726/users/98c0959d4e744b38a62c053fb80ad52f/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.752028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3d2d1-b030-47d5-95b8-88b2e874a281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=554393e62a6c4dacb08201cdf828b726, user_id=98c0959d4e744b38a62c053fb80ad52f, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:04.754787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3d2d1-b030-47d5-95b8-88b2e874a281 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:04.856771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1238/2473] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:04 2022] PUT /identity/v3/projects/554393e62a6c4dacb08201cdf828b726/users/98c0959d4e744b38a62c053fb80ad52f/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:04.873404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ade6-1919-4770-93a4-98385d71be81 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:04.873828 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ade6-1919-4770-93a4-98385d71be81 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:04.874170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ade6-1919-4770-93a4-98385d71be81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.902192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-d613ade6-1919-4770-93a4-98385d71be81 None None] MFA Rules not processed for user `98c0959d4e744b38a62c053fb80ad52f`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:04.903388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d613ade6-1919-4770-93a4-98385d71be81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:04.970249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1236/2474] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:04.976060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3d2d1-b030-47d5-95b8-88b2e874a281 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:04.997658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:05.040816 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3d2d1-b030-47d5-95b8-88b2e874a281 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:05.046038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d5e1c-b6a7-4f9d-b71f-2ed30393562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:05.046661 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d5e1c-b6a7-4f9d-b71f-2ed30393562a admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:05.046928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d5e1c-b6a7-4f9d-b71f-2ed30393562a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:05.047205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d5e1c-b6a7-4f9d-b71f-2ed30393562a admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:05.047827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d5e1c-b6a7-4f9d-b71f-2ed30393562a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:05.049809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d5e1c-b6a7-4f9d-b71f-2ed30393562a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:05.190895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1239/2475] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:24:04 2022] POST /identity/v3/projects => generated 375 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:05.193188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613ade6-1919-4770-93a4-98385d71be81 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:05.205264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613ade6-1919-4770-93a4-98385d71be81 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:05.265711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b652834-f301-41ec-9523-a7e7697dc1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:05.266254 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b652834-f301-41ec-9523-a7e7697dc1fe admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:05.266533 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b652834-f301-41ec-9523-a7e7697dc1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:05.266805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b652834-f301-41ec-9523-a7e7697dc1fe admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:05.267433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b652834-f301-41ec-9523-a7e7697dc1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:05.269383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b652834-f301-41ec-9523-a7e7697dc1fe admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:05.349248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1237/2476] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:24:05 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:05.350763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d5e1c-b6a7-4f9d-b71f-2ed30393562a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:05.362433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d5e1c-b6a7-4f9d-b71f-2ed30393562a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:05.368691 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed66fe-c26c-4cf3-9650-46275d3255e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:05.369358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed66fe-c26c-4cf3-9650-46275d3255e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:05.369643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed66fe-c26c-4cf3-9650-46275d3255e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:05.369947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed66fe-c26c-4cf3-9650-46275d3255e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:05.370701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed66fe-c26c-4cf3-9650-46275d3255e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:05.372325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed66fe-c26c-4cf3-9650-46275d3255e7 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:05.385211 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1240/2477] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:05.389357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b652834-f301-41ec-9523-a7e7697dc1fe admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:05.401110 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b652834-f301-41ec-9523-a7e7697dc1fe admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:05.406057 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79288b-d1bb-4d7b-9fe1-5609091b627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:05.406723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79288b-d1bb-4d7b-9fe1-5609091b627e admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:05.407005 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79288b-d1bb-4d7b-9fe1-5609091b627e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:05.407282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79288b-d1bb-4d7b-9fe1-5609091b627e admin admin] PATH_INFO: `/v3/projects/dd556036654d405692b1ba858332f394/users/1062a4d5419e47b5b61beab7102a8590/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:05.423646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79288b-d1bb-4d7b-9fe1-5609091b627e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd556036654d405692b1ba858332f394, user_id=1062a4d5419e47b5b61beab7102a8590, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:05.426417 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79288b-d1bb-4d7b-9fe1-5609091b627e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:05.556482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1238/2478] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:05 2022] PUT /identity/v3/projects/dd556036654d405692b1ba858332f394/users/1062a4d5419e47b5b61beab7102a8590/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:24:05.560414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed66fe-c26c-4cf3-9650-46275d3255e7 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:05.572020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed66fe-c26c-4cf3-9650-46275d3255e7 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:05.631303 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df2a0b-3422-414d-9d51-2912ae6bb03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:05.632932 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df2a0b-3422-414d-9d51-2912ae6bb03c admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:05.633226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df2a0b-3422-414d-9d51-2912ae6bb03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:05.633500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df2a0b-3422-414d-9d51-2912ae6bb03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:05.634140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df2a0b-3422-414d-9d51-2912ae6bb03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:05.635724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df2a0b-3422-414d-9d51-2912ae6bb03c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:05.645617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1241/2479] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:05.650145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79288b-d1bb-4d7b-9fe1-5609091b627e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:05.656834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:05.660561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79288b-d1bb-4d7b-9fe1-5609091b627e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:05.665426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4ef8d-e3b2-4418-9496-c524fa961068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:05.666079 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4ef8d-e3b2-4418-9496-c524fa961068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:05.666428 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4ef8d-e3b2-4418-9496-c524fa961068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:05.666666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4ef8d-e3b2-4418-9496-c524fa961068 admin admin] PATH_INFO: `/v3/projects/dd556036654d405692b1ba858332f394/users/1062a4d5419e47b5b61beab7102a8590/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:05.668595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4ef8d-e3b2-4418-9496-c524fa961068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd556036654d405692b1ba858332f394, user_id=1062a4d5419e47b5b61beab7102a8590, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:05.671283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4ef8d-e3b2-4418-9496-c524fa961068 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:05.681970 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1239/2480] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:05 2022] PUT /identity/v3/projects/dd556036654d405692b1ba858332f394/users/1062a4d5419e47b5b61beab7102a8590/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:05.693674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cfb8f-d321-429e-9585-6ceaf99f1ca1 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:05.694190 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cfb8f-d321-429e-9585-6ceaf99f1ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:05.695264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cfb8f-d321-429e-9585-6ceaf99f1ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:05.723183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-b16cfb8f-d321-429e-9585-6ceaf99f1ca1 None None] MFA Rules not processed for user `1062a4d5419e47b5b61beab7102a8590`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:05.724483 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b16cfb8f-d321-429e-9585-6ceaf99f1ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:05.797536 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1242/2481] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:05.809342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4ef8d-e3b2-4418-9496-c524fa961068 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:05.811029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b8d4ef8d-e3b2-4418-9496-c524fa961068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:05.870026 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4ef8d-e3b2-4418-9496-c524fa961068 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:05.873901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130ca14-1a1d-40a0-94c0-aea6c4ba758b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:05.874502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ca14-1a1d-40a0-94c0-aea6c4ba758b service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:05.874778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ca14-1a1d-40a0-94c0-aea6c4ba758b service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:05.875046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ca14-1a1d-40a0-94c0-aea6c4ba758b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:05.937983 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130ca14-1a1d-40a0-94c0-aea6c4ba758b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:05.939452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130ca14-1a1d-40a0-94c0-aea6c4ba758b service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:05.949726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1240/2482] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:06.634712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cfb8f-d321-429e-9585-6ceaf99f1ca1 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:06.636170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b16cfb8f-d321-429e-9585-6ceaf99f1ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:06.648433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cfb8f-d321-429e-9585-6ceaf99f1ca1 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:06.652201 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9db3e-4ba1-4ae0-b924-e36d71fc023b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:06.652838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9db3e-4ba1-4ae0-b924-e36d71fc023b service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:06.653150 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9db3e-4ba1-4ae0-b924-e36d71fc023b service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:06.653605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9db3e-4ba1-4ae0-b924-e36d71fc023b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:06.655352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-abf9db3e-4ba1-4ae0-b924-e36d71fc023b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:06.667459 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:06.718895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9db3e-4ba1-4ae0-b924-e36d71fc023b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:06.720343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9db3e-4ba1-4ae0-b924-e36d71fc023b service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:06.730595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1243/2483] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:06.978982 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac6c5c-def8-4c2e-b1e9-6d6df0431a94 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:06.979541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac6c5c-def8-4c2e-b1e9-6d6df0431a94 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:06.979541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac6c5c-def8-4c2e-b1e9-6d6df0431a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:06.999042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-92ac6c5c-def8-4c2e-b1e9-6d6df0431a94 None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:07.000264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-92ac6c5c-def8-4c2e-b1e9-6d6df0431a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:07.057480 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1241/2484] 10.222.0.30 () {58 vars in 967 bytes} [Wed Aug 24 10:24:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:07.070720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9db3e-4ba1-4ae0-b924-e36d71fc023b service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:07.072466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-abf9db3e-4ba1-4ae0-b924-e36d71fc023b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:07.155982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9db3e-4ba1-4ae0-b924-e36d71fc023b service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:07.162276 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57304b73-f29c-40f0-949c-7ad6ff3c1a47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:07.162654 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57304b73-f29c-40f0-949c-7ad6ff3c1a47 service swift] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:07.163718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57304b73-f29c-40f0-949c-7ad6ff3c1a47 service swift] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:07.164231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57304b73-f29c-40f0-949c-7ad6ff3c1a47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:07.173776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57304b73-f29c-40f0-949c-7ad6ff3c1a47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:07.175377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57304b73-f29c-40f0-949c-7ad6ff3c1a47 service swift] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:07.184403 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1244/2485] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:24:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:24:08.300909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac6c5c-def8-4c2e-b1e9-6d6df0431a94 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:08.310923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac6c5c-def8-4c2e-b1e9-6d6df0431a94 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:08.314472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ae1d2-7bdd-435a-8e22-73a22e2d5fff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:08.315067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ae1d2-7bdd-435a-8e22-73a22e2d5fff service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:08.315343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ae1d2-7bdd-435a-8e22-73a22e2d5fff service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:08.315612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ae1d2-7bdd-435a-8e22-73a22e2d5fff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:08.368351 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:08.378778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ae1d2-7bdd-435a-8e22-73a22e2d5fff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:08.380398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ae1d2-7bdd-435a-8e22-73a22e2d5fff service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:08.389886 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1242/2486] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:11.748304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57304b73-f29c-40f0-949c-7ad6ff3c1a47 service swift] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:11.757625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57304b73-f29c-40f0-949c-7ad6ff3c1a47 service swift] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:11.761851 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db88b0-7789-4655-be22-b7000866b4dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:11.762444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db88b0-7789-4655-be22-b7000866b4dd service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:11.762722 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db88b0-7789-4655-be22-b7000866b4dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:11.763000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db88b0-7789-4655-be22-b7000866b4dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:11.772641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db88b0-7789-4655-be22-b7000866b4dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:11.775116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db88b0-7789-4655-be22-b7000866b4dd service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:11.785038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1245/2487] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:24:12.521997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ae1d2-7bdd-435a-8e22-73a22e2d5fff service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:12.523506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b92ae1d2-7bdd-435a-8e22-73a22e2d5fff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:12.589375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ae1d2-7bdd-435a-8e22-73a22e2d5fff service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:12.594265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:12.594889 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:12.595279 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:12.595563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] PATH_INFO: `/v3/users/2dbe7a9833e64b6b8fcad4cc1298d4e6` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:12.596751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dbe7a9833e64b6b8fcad4cc1298d4e6)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:12.598729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:12.865441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dbe7a9833e64b6b8fcad4cc1298d4e6'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:12.866136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dbe7a9833e64b6b8fcad4cc1298d4e6'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:12.874283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dbe7a9833e64b6b8fcad4cc1298d4e6'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:12.887619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dbe7a9833e64b6b8fcad4cc1298d4e6'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:13.021935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dbe7a9833e64b6b8fcad4cc1298d4e6'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:13.030984 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1243/2488] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:12 2022] DELETE /identity/v3/users/2dbe7a9833e64b6b8fcad4cc1298d4e6 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:13.035367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e759e-2498-4864-a83d-8843d801d6c4 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:13.035862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e759e-2498-4864-a83d-8843d801d6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:13.036109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e759e-2498-4864-a83d-8843d801d6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:13.057577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-366e759e-2498-4864-a83d-8843d801d6c4 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:13.058829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-366e759e-2498-4864-a83d-8843d801d6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:13.133290 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1246/2489] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:13.143278 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:13.144765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:13.215579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6e0fc-359e-4c3e-b1e7-f57b35d05128 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:13.220358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe672a4f-4a96-49d5-97fd-d6929d727252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:13.221028 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe672a4f-4a96-49d5-97fd-d6929d727252 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:13.221310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe672a4f-4a96-49d5-97fd-d6929d727252 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:13.221582 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe672a4f-4a96-49d5-97fd-d6929d727252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:13.234196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe672a4f-4a96-49d5-97fd-d6929d727252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:13.236008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe672a4f-4a96-49d5-97fd-d6929d727252 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:13.247901 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1244/2490] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:13.704581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e759e-2498-4864-a83d-8843d801d6c4 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:13.705743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-366e759e-2498-4864-a83d-8843d801d6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:13.718542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e759e-2498-4864-a83d-8843d801d6c4 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:13.725477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:13.726151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:13.726437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:13.726716 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] PATH_INFO: `/v3/projects/ec5fe346cd4e4751b5e407cc67d37f2c` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:13.728419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec5fe346cd4e4751b5e407cc67d37f2c)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:13.730567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:13.764154 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:13.875642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec5fe346cd4e4751b5e407cc67d37f2c', 'request_id': 'req-69f0f358-c655-4c06-b3bf-ba3cacf08c26'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:13.883740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec5fe346cd4e4751b5e407cc67d37f2c', 'request_id': 'req-69f0f358-c655-4c06-b3bf-ba3cacf08c26'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:13.893572 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec5fe346cd4e4751b5e407cc67d37f2c', 'request_id': 'req-69f0f358-c655-4c06-b3bf-ba3cacf08c26'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:13.971027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] The token cache is being invalidate because project ec5fe346cd4e4751b5e407cc67d37f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:13.971027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:13.971701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:13.976358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1247/2491] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:13 2022] DELETE /identity/v3/projects/ec5fe346cd4e4751b5e407cc67d37f2c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:13.978846 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe672a4f-4a96-49d5-97fd-d6929d727252 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:13.990802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe672a4f-4a96-49d5-97fd-d6929d727252 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:14.002467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:14.053399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538e5f8-696a-4ea3-847d-8695d09ce566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:14.054039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538e5f8-696a-4ea3-847d-8695d09ce566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:14.054325 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538e5f8-696a-4ea3-847d-8695d09ce566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:14.054598 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538e5f8-696a-4ea3-847d-8695d09ce566 admin admin] PATH_INFO: `/v3/users/cf30ff2f64a34489a86bc1ec2e88e7b2` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:14.055718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538e5f8-696a-4ea3-847d-8695d09ce566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf30ff2f64a34489a86bc1ec2e88e7b2)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:14.058202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538e5f8-696a-4ea3-847d-8695d09ce566 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:14.373260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7538e5f8-696a-4ea3-847d-8695d09ce566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf30ff2f64a34489a86bc1ec2e88e7b2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:14.374010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7538e5f8-696a-4ea3-847d-8695d09ce566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf30ff2f64a34489a86bc1ec2e88e7b2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:14.381377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7538e5f8-696a-4ea3-847d-8695d09ce566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf30ff2f64a34489a86bc1ec2e88e7b2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:14.394808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7538e5f8-696a-4ea3-847d-8695d09ce566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf30ff2f64a34489a86bc1ec2e88e7b2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:14.502154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7538e5f8-696a-4ea3-847d-8695d09ce566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf30ff2f64a34489a86bc1ec2e88e7b2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:14.508461 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1245/2492] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:13 2022] DELETE /identity/v3/users/cf30ff2f64a34489a86bc1ec2e88e7b2 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:15.305088 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:15.306828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:15.325215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0f358-c655-4c06-b3bf-ba3cacf08c26 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:15.397038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:15.397038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:15.397038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:15.397402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] PATH_INFO: `/v3/projects/4373e2095dfe450c8a05a813dd409c2f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:15.399095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4373e2095dfe450c8a05a813dd409c2f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:15.401357 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:15.534297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4373e2095dfe450c8a05a813dd409c2f', 'request_id': 'req-767bfb69-6d27-449d-9595-c6c65706f0c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:15.541215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4373e2095dfe450c8a05a813dd409c2f', 'request_id': 'req-767bfb69-6d27-449d-9595-c6c65706f0c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:15.550037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4373e2095dfe450c8a05a813dd409c2f', 'request_id': 'req-767bfb69-6d27-449d-9595-c6c65706f0c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:15.673480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] The token cache is being invalidate because project 4373e2095dfe450c8a05a813dd409c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:15.673778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:15.674451 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-767bfb69-6d27-449d-9595-c6c65706f0c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:15.676058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1248/2493] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:15 2022] DELETE /identity/v3/projects/4373e2095dfe450c8a05a813dd409c2f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:15.681053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e044299-90c7-4c0b-99fd-ddf1d08a8dea None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:15.681788 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e044299-90c7-4c0b-99fd-ddf1d08a8dea None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:15.681788 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e044299-90c7-4c0b-99fd-ddf1d08a8dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:15.702565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-3e044299-90c7-4c0b-99fd-ddf1d08a8dea None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:15.703978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3e044299-90c7-4c0b-99fd-ddf1d08a8dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:15.768653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1246/2494] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:15.776779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2764910-d00e-4bd0-ad71-8a93a148ae20 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:15.777090 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2764910-d00e-4bd0-ad71-8a93a148ae20 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:15.777365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2764910-d00e-4bd0-ad71-8a93a148ae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:15.798319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c2764910-d00e-4bd0-ad71-8a93a148ae20 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:15.800335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c2764910-d00e-4bd0-ad71-8a93a148ae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:15.809115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1249/2495] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:15.813452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e044299-90c7-4c0b-99fd-ddf1d08a8dea None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:15.824035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e044299-90c7-4c0b-99fd-ddf1d08a8dea None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:15.828648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9f9d8-414c-413e-9582-6c14f271b10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:15.829310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9f9d8-414c-413e-9582-6c14f271b10f admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:15.829589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9f9d8-414c-413e-9582-6c14f271b10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:15.829864 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9f9d8-414c-413e-9582-6c14f271b10f admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:15.830541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9f9d8-414c-413e-9582-6c14f271b10f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:24:15.830843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9f9d8-414c-413e-9582-6c14f271b10f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:15.832466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9f9d8-414c-413e-9582-6c14f271b10f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:15.846517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1247/2496] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:24:15.851162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2764910-d00e-4bd0-ad71-8a93a148ae20 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:15.862040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2764910-d00e-4bd0-ad71-8a93a148ae20 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:15.866920 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48767203-943f-47f1-b342-4339df813aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:15.867567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48767203-943f-47f1-b342-4339df813aaa admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:15.867872 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48767203-943f-47f1-b342-4339df813aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:15.868170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48767203-943f-47f1-b342-4339df813aaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:15.868893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48767203-943f-47f1-b342-4339df813aaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:15.871123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48767203-943f-47f1-b342-4339df813aaa admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:15.972687 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1250/2497] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:24:15 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:15.977059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9f9d8-414c-413e-9582-6c14f271b10f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:15.988327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9f9d8-414c-413e-9582-6c14f271b10f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:16.001075 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:16.048249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6d53-b1f3-4c70-bb1b-217c9c5f3d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:16.048955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6d53-b1f3-4c70-bb1b-217c9c5f3d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:16.049467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6d53-b1f3-4c70-bb1b-217c9c5f3d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:16.050479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6d53-b1f3-4c70-bb1b-217c9c5f3d7f admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:16.051117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba6d53-b1f3-4c70-bb1b-217c9c5f3d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:16.053178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba6d53-b1f3-4c70-bb1b-217c9c5f3d7f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:16.188311 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1248/2498] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:24:15 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:16.192543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48767203-943f-47f1-b342-4339df813aaa admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:16.204252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48767203-943f-47f1-b342-4339df813aaa admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:16.209082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730cfd6-a69d-4a91-b53d-671cd135a664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:16.209701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730cfd6-a69d-4a91-b53d-671cd135a664 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:16.209983 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730cfd6-a69d-4a91-b53d-671cd135a664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:16.210254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730cfd6-a69d-4a91-b53d-671cd135a664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:16.210895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730cfd6-a69d-4a91-b53d-671cd135a664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:16.213431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730cfd6-a69d-4a91-b53d-671cd135a664 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:16.215993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:16.225055 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1251/2499] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:16.229768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6d53-b1f3-4c70-bb1b-217c9c5f3d7f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:16.240319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6d53-b1f3-4c70-bb1b-217c9c5f3d7f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:16.245395 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0980a38-14cd-4097-90dd-40af8c2623a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:16.245994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980a38-14cd-4097-90dd-40af8c2623a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:16.246274 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980a38-14cd-4097-90dd-40af8c2623a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:16.246547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980a38-14cd-4097-90dd-40af8c2623a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:16.247210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0980a38-14cd-4097-90dd-40af8c2623a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:16.248843 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0980a38-14cd-4097-90dd-40af8c2623a9 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:16.262541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1249/2500] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:24:16.266793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730cfd6-a69d-4a91-b53d-671cd135a664 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:16.278894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730cfd6-a69d-4a91-b53d-671cd135a664 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:16.283593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9aea11-dccf-4f11-baff-b1262ac6dc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:16.284268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aea11-dccf-4f11-baff-b1262ac6dc7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:16.284548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aea11-dccf-4f11-baff-b1262ac6dc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:16.284821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aea11-dccf-4f11-baff-b1262ac6dc7b admin admin] PATH_INFO: `/v3/projects/3456d6c9778140e486b78e739a0588e2/users/293fbfbe348f4b73ad3dc9c0633d9c74/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:16.300581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9aea11-dccf-4f11-baff-b1262ac6dc7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3456d6c9778140e486b78e739a0588e2, user_id=293fbfbe348f4b73ad3dc9c0633d9c74, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:16.303268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9aea11-dccf-4f11-baff-b1262ac6dc7b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:16.376586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1252/2501] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:16 2022] PUT /identity/v3/projects/3456d6c9778140e486b78e739a0588e2/users/293fbfbe348f4b73ad3dc9c0633d9c74/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:16.381880 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013850b-e3a0-4a04-906f-4654dc2dbfcb None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:16.382197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013850b-e3a0-4a04-906f-4654dc2dbfcb None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:16.382486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013850b-e3a0-4a04-906f-4654dc2dbfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:16.403208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-8013850b-e3a0-4a04-906f-4654dc2dbfcb None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:16.404436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8013850b-e3a0-4a04-906f-4654dc2dbfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:16.466868 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1250/2502] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:16.476792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9aea11-dccf-4f11-baff-b1262ac6dc7b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:16.478592 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-be9aea11-dccf-4f11-baff-b1262ac6dc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:16.537252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9aea11-dccf-4f11-baff-b1262ac6dc7b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:16.541227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa1306-abcb-4d48-9185-7fb5feae6c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:16.541830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa1306-abcb-4d48-9185-7fb5feae6c38 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:16.542112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa1306-abcb-4d48-9185-7fb5feae6c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:16.542393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa1306-abcb-4d48-9185-7fb5feae6c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:16.551715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa1306-abcb-4d48-9185-7fb5feae6c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:16.553179 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa1306-abcb-4d48-9185-7fb5feae6c38 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:16.564455 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1253/2503] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:17.874055 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013850b-e3a0-4a04-906f-4654dc2dbfcb None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:17.876054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8013850b-e3a0-4a04-906f-4654dc2dbfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:17.892144 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013850b-e3a0-4a04-906f-4654dc2dbfcb None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:17.897232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2012f-f33f-44bb-9780-540f19e12b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:17.897910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2012f-f33f-44bb-9780-540f19e12b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:17.898212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2012f-f33f-44bb-9780-540f19e12b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:17.898494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2012f-f33f-44bb-9780-540f19e12b5a admin admin] PATH_INFO: `/v3/users/76181228273f4bbd85a5aabb8ef555a5` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:17.900372 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2012f-f33f-44bb-9780-540f19e12b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=76181228273f4bbd85a5aabb8ef555a5)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:17.902319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2012f-f33f-44bb-9780-540f19e12b5a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:18.185808 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d778750-f778-4b3b-93e1-a6f3de15b899 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:18.186495 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d778750-f778-4b3b-93e1-a6f3de15b899 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:18.186495 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d778750-f778-4b3b-93e1-a6f3de15b899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:18.191256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-22d2012f-f33f-44bb-9780-540f19e12b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76181228273f4bbd85a5aabb8ef555a5'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:18.192177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-22d2012f-f33f-44bb-9780-540f19e12b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76181228273f4bbd85a5aabb8ef555a5'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:18.207472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-22d2012f-f33f-44bb-9780-540f19e12b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76181228273f4bbd85a5aabb8ef555a5'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:18.219302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1d778750-f778-4b3b-93e1-a6f3de15b899 None None] MFA Rules not processed for user `293fbfbe348f4b73ad3dc9c0633d9c74`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:18.220681 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1d778750-f778-4b3b-93e1-a6f3de15b899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:18.224328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-22d2012f-f33f-44bb-9780-540f19e12b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76181228273f4bbd85a5aabb8ef555a5'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:18.311272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1254/2504] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:18.316059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d778750-f778-4b3b-93e1-a6f3de15b899 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:18.329145 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d778750-f778-4b3b-93e1-a6f3de15b899 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:18.333993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef6f82-f8c4-4bc6-bc53-65883baae0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:18.334639 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef6f82-f8c4-4bc6-bc53-65883baae0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:18.334921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef6f82-f8c4-4bc6-bc53-65883baae0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:18.335199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef6f82-f8c4-4bc6-bc53-65883baae0c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:18.335846 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef6f82-f8c4-4bc6-bc53-65883baae0c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:18.338223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef6f82-f8c4-4bc6-bc53-65883baae0c2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:18.427413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-22d2012f-f33f-44bb-9780-540f19e12b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76181228273f4bbd85a5aabb8ef555a5'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:18.435002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1251/2505] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:17 2022] DELETE /identity/v3/users/76181228273f4bbd85a5aabb8ef555a5 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:18.439151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832474-b580-41a0-814c-64c170ea97e6 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:18.439442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832474-b580-41a0-814c-64c170ea97e6 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:18.439804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832474-b580-41a0-814c-64c170ea97e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:18.461402 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-04832474-b580-41a0-814c-64c170ea97e6 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:18.462811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-04832474-b580-41a0-814c-64c170ea97e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:18.466996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:18.508946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1255/2506] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:24:18 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:24:18.514129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef6f82-f8c4-4bc6-bc53-65883baae0c2 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:18.517827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-13ef6f82-f8c4-4bc6-bc53-65883baae0c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:18.546735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1252/2507] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:18.557275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04832474-b580-41a0-814c-64c170ea97e6 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:18.558851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-04832474-b580-41a0-814c-64c170ea97e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:18.584005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:18.610757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef6f82-f8c4-4bc6-bc53-65883baae0c2 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:18.615990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002027e-6ab1-49d0-af7e-109df548cb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:18.616442 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002027e-6ab1-49d0-af7e-109df548cb86 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:18.617256 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002027e-6ab1-49d0-af7e-109df548cb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:18.617544 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002027e-6ab1-49d0-af7e-109df548cb86 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:18.618833 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002027e-6ab1-49d0-af7e-109df548cb86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:18.621453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002027e-6ab1-49d0-af7e-109df548cb86 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:18.638719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04832474-b580-41a0-814c-64c170ea97e6 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:18.691547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d00d30-01a6-4cd8-9326-12d404b27577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:18.692190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d00d30-01a6-4cd8-9326-12d404b27577 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:18.692469 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d00d30-01a6-4cd8-9326-12d404b27577 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:18.692740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d00d30-01a6-4cd8-9326-12d404b27577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:18.702912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d00d30-01a6-4cd8-9326-12d404b27577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:18.704457 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d00d30-01a6-4cd8-9326-12d404b27577 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:18.715180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1253/2508] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:24:18.767521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1256/2509] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:24:18 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:18.768851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d00d30-01a6-4cd8-9326-12d404b27577 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:18.780430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d00d30-01a6-4cd8-9326-12d404b27577 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:18.786292 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40033fa5-81c1-4472-8344-931f50123f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:18.787095 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40033fa5-81c1-4472-8344-931f50123f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:18.787380 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40033fa5-81c1-4472-8344-931f50123f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:18.787651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40033fa5-81c1-4472-8344-931f50123f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:18.788307 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40033fa5-81c1-4472-8344-931f50123f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:18.790002 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40033fa5-81c1-4472-8344-931f50123f7e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:18.805419 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1254/2510] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:18.807620 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002027e-6ab1-49d0-af7e-109df548cb86 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:18.830093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002027e-6ab1-49d0-af7e-109df548cb86 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:18.835698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50b7d7-fe39-4cd3-b8ef-d103dfcbe339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:18.836398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b7d7-fe39-4cd3-b8ef-d103dfcbe339 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:18.836643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b7d7-fe39-4cd3-b8ef-d103dfcbe339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:18.837023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b7d7-fe39-4cd3-b8ef-d103dfcbe339 admin admin] PATH_INFO: `/v3/projects/4d309a4f701e40b197f5650df92c703b/users/128aa8503268455d8a735b5129362b40/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:18.854884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50b7d7-fe39-4cd3-b8ef-d103dfcbe339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d309a4f701e40b197f5650df92c703b, user_id=128aa8503268455d8a735b5129362b40, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:18.857604 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50b7d7-fe39-4cd3-b8ef-d103dfcbe339 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:18.979048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1257/2511] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:18 2022] PUT /identity/v3/projects/4d309a4f701e40b197f5650df92c703b/users/128aa8503268455d8a735b5129362b40/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:18.993931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40033fa5-81c1-4472-8344-931f50123f7e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:19.002706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40033fa5-81c1-4472-8344-931f50123f7e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:19.078764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf9530-9b3a-4a02-99dc-291f4dc35b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:19.079702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf9530-9b3a-4a02-99dc-291f4dc35b4b admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:19.079999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf9530-9b3a-4a02-99dc-291f4dc35b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:19.080265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf9530-9b3a-4a02-99dc-291f4dc35b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:19.080973 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf9530-9b3a-4a02-99dc-291f4dc35b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:19.082623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf9530-9b3a-4a02-99dc-291f4dc35b4b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:19.093855 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1255/2512] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:19.098018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50b7d7-fe39-4cd3-b8ef-d103dfcbe339 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:19.119396 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50b7d7-fe39-4cd3-b8ef-d103dfcbe339 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:19.123921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bacb4-b281-472d-9b76-53506c519b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:19.124695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bacb4-b281-472d-9b76-53506c519b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:19.125111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bacb4-b281-472d-9b76-53506c519b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:19.125472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bacb4-b281-472d-9b76-53506c519b56 admin admin] PATH_INFO: `/v3/projects/4d309a4f701e40b197f5650df92c703b/users/128aa8503268455d8a735b5129362b40/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:19.129375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bacb4-b281-472d-9b76-53506c519b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d309a4f701e40b197f5650df92c703b, user_id=128aa8503268455d8a735b5129362b40, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:19.133257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bacb4-b281-472d-9b76-53506c519b56 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:19.151841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1258/2513] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:19 2022] PUT /identity/v3/projects/4d309a4f701e40b197f5650df92c703b/users/128aa8503268455d8a735b5129362b40/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:19.352234 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf9530-9b3a-4a02-99dc-291f4dc35b4b admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:19.354760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-79bf9530-9b3a-4a02-99dc-291f4dc35b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:19.370093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf9530-9b3a-4a02-99dc-291f4dc35b4b admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:19.432225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:19.432871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:19.433157 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:19.433430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] PATH_INFO: `/v3/projects/0d66e5f6205f4c6bb8dc4e50a94871a3` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:19.434547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d66e5f6205f4c6bb8dc4e50a94871a3)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:19.436444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:19.473386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:19.584033 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d66e5f6205f4c6bb8dc4e50a94871a3', 'request_id': 'req-d19f42ba-1490-4968-a53a-a31c7175db7d'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:19.592046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d66e5f6205f4c6bb8dc4e50a94871a3', 'request_id': 'req-d19f42ba-1490-4968-a53a-a31c7175db7d'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:19.701011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d66e5f6205f4c6bb8dc4e50a94871a3', 'request_id': 'req-d19f42ba-1490-4968-a53a-a31c7175db7d'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:19.718461 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] The token cache is being invalidate because project 0d66e5f6205f4c6bb8dc4e50a94871a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:19.718461 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:19.718461 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d19f42ba-1490-4968-a53a-a31c7175db7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:19.719125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1256/2514] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:19 2022] DELETE /identity/v3/projects/0d66e5f6205f4c6bb8dc4e50a94871a3 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:19.723046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bacb4-b281-472d-9b76-53506c519b56 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:19.724804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e71bacb4-b281-472d-9b76-53506c519b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:19.738782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bacb4-b281-472d-9b76-53506c519b56 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:19.816295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:19.817091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:19.817448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:19.817825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] PATH_INFO: `/v3/users/98c0959d4e744b38a62c053fb80ad52f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:19.819378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] RBAC: Authorizing `identity:delete_user(user_id=98c0959d4e744b38a62c053fb80ad52f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:19.821982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:20.243476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98c0959d4e744b38a62c053fb80ad52f'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:20.244229 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98c0959d4e744b38a62c053fb80ad52f'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:20.255542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98c0959d4e744b38a62c053fb80ad52f'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:20.257388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98c0959d4e744b38a62c053fb80ad52f'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:20.404977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98c0959d4e744b38a62c053fb80ad52f'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:20.440205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1259/2515] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:19 2022] DELETE /identity/v3/users/98c0959d4e744b38a62c053fb80ad52f => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:24:20.956566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22a5d7-ea54-40a9-9297-aa5b6ce63ea0 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:20.957250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22a5d7-ea54-40a9-9297-aa5b6ce63ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:20.957250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22a5d7-ea54-40a9-9297-aa5b6ce63ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:20.957825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:20.959368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:20.991999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-7b22a5d7-ea54-40a9-9297-aa5b6ce63ea0 None None] MFA Rules not processed for user `128aa8503268455d8a735b5129362b40`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:20.993920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7b22a5d7-ea54-40a9-9297-aa5b6ce63ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:20.996884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:21.069701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e201ff-e5e9-4110-a387-542097ddd29e admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:21.075010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:21.075901 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:21.076378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:21.076655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] PATH_INFO: `/v3/projects/554393e62a6c4dacb08201cdf828b726` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:21.077863 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] RBAC: Authorizing `identity:delete_project(project_id=554393e62a6c4dacb08201cdf828b726)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:21.079935 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:21.096765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1257/2516] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:21.116589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22a5d7-ea54-40a9-9297-aa5b6ce63ea0 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:21.116589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7b22a5d7-ea54-40a9-9297-aa5b6ce63ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:21.123576 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:21.176080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22a5d7-ea54-40a9-9297-aa5b6ce63ea0 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:21.179896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a352541-46fb-41e4-a892-3050e2199b2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:21.180499 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a352541-46fb-41e4-a892-3050e2199b2a service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:21.180818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a352541-46fb-41e4-a892-3050e2199b2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:21.181238 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a352541-46fb-41e4-a892-3050e2199b2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:21.185103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8a352541-46fb-41e4-a892-3050e2199b2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:21.241815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '554393e62a6c4dacb08201cdf828b726', 'request_id': 'req-0075bf6a-424d-4d83-8c4a-a0634261965d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:21.247966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '554393e62a6c4dacb08201cdf828b726', 'request_id': 'req-0075bf6a-424d-4d83-8c4a-a0634261965d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:21.248166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a352541-46fb-41e4-a892-3050e2199b2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:21.249521 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a352541-46fb-41e4-a892-3050e2199b2a service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:21.254336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '554393e62a6c4dacb08201cdf828b726', 'request_id': 'req-0075bf6a-424d-4d83-8c4a-a0634261965d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:21.258868 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1258/2517] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:21.381818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] The token cache is being invalidate because project 554393e62a6c4dacb08201cdf828b726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:21.382114 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:21.382767 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:21.384526 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1260/2518] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:20 2022] DELETE /identity/v3/projects/554393e62a6c4dacb08201cdf828b726 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:21.394262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a352541-46fb-41e4-a892-3050e2199b2a service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:21.406155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a352541-46fb-41e4-a892-3050e2199b2a service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:21.466493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bfba3-c4ac-4e62-89ce-bd9c8f264c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:21.467142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bfba3-c4ac-4e62-89ce-bd9c8f264c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:21.467425 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bfba3-c4ac-4e62-89ce-bd9c8f264c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:21.467709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bfba3-c4ac-4e62-89ce-bd9c8f264c17 admin admin] PATH_INFO: `/v3/users/1062a4d5419e47b5b61beab7102a8590` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:21.468892 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36bfba3-c4ac-4e62-89ce-bd9c8f264c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1062a4d5419e47b5b61beab7102a8590)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:21.470952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36bfba3-c4ac-4e62-89ce-bd9c8f264c17 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:21.847558 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b36bfba3-c4ac-4e62-89ce-bd9c8f264c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1062a4d5419e47b5b61beab7102a8590'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:21.848377 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b36bfba3-c4ac-4e62-89ce-bd9c8f264c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1062a4d5419e47b5b61beab7102a8590'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:21.855922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b36bfba3-c4ac-4e62-89ce-bd9c8f264c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1062a4d5419e47b5b61beab7102a8590'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:21.867878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b36bfba3-c4ac-4e62-89ce-bd9c8f264c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1062a4d5419e47b5b61beab7102a8590'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:22.004831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-b36bfba3-c4ac-4e62-89ce-bd9c8f264c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1062a4d5419e47b5b61beab7102a8590'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:22.019129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1259/2519] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:21 2022] DELETE /identity/v3/users/1062a4d5419e47b5b61beab7102a8590 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:22.509411 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:22.511772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:22.531152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075bf6a-424d-4d83-8c4a-a0634261965d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:22.599256 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:22.599912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:22.600188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:22.600445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] PATH_INFO: `/v3/projects/dd556036654d405692b1ba858332f394` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:22.601593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd556036654d405692b1ba858332f394)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:22.604214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:22.834147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd556036654d405692b1ba858332f394', 'request_id': 'req-97fddb38-257b-4ee1-85fa-3c502cdb49a7'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:22.834147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd556036654d405692b1ba858332f394', 'request_id': 'req-97fddb38-257b-4ee1-85fa-3c502cdb49a7'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:22.839858 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd556036654d405692b1ba858332f394', 'request_id': 'req-97fddb38-257b-4ee1-85fa-3c502cdb49a7'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:23.066497 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] The token cache is being invalidate because project dd556036654d405692b1ba858332f394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:23.066787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:23.067440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-97fddb38-257b-4ee1-85fa-3c502cdb49a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:23.068991 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1261/2520] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:22 2022] DELETE /identity/v3/projects/dd556036654d405692b1ba858332f394 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:23.074917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4768d-4832-4a8d-a426-6a22688a19b0 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:23.075431 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4768d-4832-4a8d-a426-6a22688a19b0 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:23.075528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4768d-4832-4a8d-a426-6a22688a19b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:23.096237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-69b4768d-4832-4a8d-a426-6a22688a19b0 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:23.097407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-69b4768d-4832-4a8d-a426-6a22688a19b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:23.161282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1260/2521] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:23.169301 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62404e-79ff-448f-9397-b4c15d6d5d94 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:23.169589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62404e-79ff-448f-9397-b4c15d6d5d94 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:23.169866 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62404e-79ff-448f-9397-b4c15d6d5d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:23.188814 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-db62404e-79ff-448f-9397-b4c15d6d5d94 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:23.190032 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-db62404e-79ff-448f-9397-b4c15d6d5d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:23.198749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1262/2522] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:23.203140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4768d-4832-4a8d-a426-6a22688a19b0 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:23.209963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:23.213807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4768d-4832-4a8d-a426-6a22688a19b0 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:23.218317 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45430b9-c782-475d-acf2-a0ea7d0f153a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:23.218915 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45430b9-c782-475d-acf2-a0ea7d0f153a admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:23.219190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45430b9-c782-475d-acf2-a0ea7d0f153a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:23.219465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45430b9-c782-475d-acf2-a0ea7d0f153a admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:23.220102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45430b9-c782-475d-acf2-a0ea7d0f153a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:24:23.220401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45430b9-c782-475d-acf2-a0ea7d0f153a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:23.222054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45430b9-c782-475d-acf2-a0ea7d0f153a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:23.234324 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1261/2523] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:24:23.239224 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62404e-79ff-448f-9397-b4c15d6d5d94 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:23.248369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62404e-79ff-448f-9397-b4c15d6d5d94 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:23.253079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ed9f2-2e64-4af6-aeae-7516265ac38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:23.253670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ed9f2-2e64-4af6-aeae-7516265ac38d admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:23.253986 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ed9f2-2e64-4af6-aeae-7516265ac38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:23.254258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ed9f2-2e64-4af6-aeae-7516265ac38d admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:23.254928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ed9f2-2e64-4af6-aeae-7516265ac38d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:23.256416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:23.256806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ed9f2-2e64-4af6-aeae-7516265ac38d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:23.406443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1263/2524] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:24:23 2022] POST /identity/v3/projects => generated 365 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:23.410212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45430b9-c782-475d-acf2-a0ea7d0f153a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:23.421793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45430b9-c782-475d-acf2-a0ea7d0f153a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:23.479956 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa961e22-5068-496e-8835-fdf94b162ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:23.480626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa961e22-5068-496e-8835-fdf94b162ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:23.480933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa961e22-5068-496e-8835-fdf94b162ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:23.481206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa961e22-5068-496e-8835-fdf94b162ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:23.481815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa961e22-5068-496e-8835-fdf94b162ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:23.483707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa961e22-5068-496e-8835-fdf94b162ee6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:23.639828 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1262/2525] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:24:23 2022] POST /identity/v3/users => generated 283 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:24:23.643941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ed9f2-2e64-4af6-aeae-7516265ac38d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:23.654586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ed9f2-2e64-4af6-aeae-7516265ac38d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:23.659183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ea05-8cb9-433a-ac37-2c2892fb4b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:23.659787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ea05-8cb9-433a-ac37-2c2892fb4b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:23.660050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ea05-8cb9-433a-ac37-2c2892fb4b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:23.660426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ea05-8cb9-433a-ac37-2c2892fb4b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:23.661099 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926ea05-8cb9-433a-ac37-2c2892fb4b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:23.662708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926ea05-8cb9-433a-ac37-2c2892fb4b25 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:23.672824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1264/2526] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:23.676752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa961e22-5068-496e-8835-fdf94b162ee6 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:23.687875 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa961e22-5068-496e-8835-fdf94b162ee6 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:23.692400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c083b-3f5b-4b86-ba58-d62dbf5db084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:23.693011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c083b-3f5b-4b86-ba58-d62dbf5db084 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:23.693318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c083b-3f5b-4b86-ba58-d62dbf5db084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:23.693566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c083b-3f5b-4b86-ba58-d62dbf5db084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:23.694190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c083b-3f5b-4b86-ba58-d62dbf5db084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:23.695768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c083b-3f5b-4b86-ba58-d62dbf5db084 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:23.706938 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1263/2527] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:23.711226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ea05-8cb9-433a-ac37-2c2892fb4b25 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:23.721512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ea05-8cb9-433a-ac37-2c2892fb4b25 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:23.726245 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9af109-343c-44b9-a3a2-85ad761319ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:23.726883 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9af109-343c-44b9-a3a2-85ad761319ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:23.727169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9af109-343c-44b9-a3a2-85ad761319ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:23.727444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9af109-343c-44b9-a3a2-85ad761319ef admin admin] PATH_INFO: `/v3/projects/9cf504ae1ab34f86923979670ac1c013/users/fde2b88311be4b58b1cf9efc1806e2c1/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:23.745835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9af109-343c-44b9-a3a2-85ad761319ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf504ae1ab34f86923979670ac1c013, user_id=fde2b88311be4b58b1cf9efc1806e2c1, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:23.748929 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9af109-343c-44b9-a3a2-85ad761319ef admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:23.886823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1265/2528] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:23 2022] PUT /identity/v3/projects/9cf504ae1ab34f86923979670ac1c013/users/fde2b88311be4b58b1cf9efc1806e2c1/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:24:23.898781 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843cb51-470c-4dc9-a7f5-be17b10485cb None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:23.899072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843cb51-470c-4dc9-a7f5-be17b10485cb None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:23.899383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843cb51-470c-4dc9-a7f5-be17b10485cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:23.930060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0843cb51-470c-4dc9-a7f5-be17b10485cb None None] MFA Rules not processed for user `fde2b88311be4b58b1cf9efc1806e2c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:23.931310 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0843cb51-470c-4dc9-a7f5-be17b10485cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:23.976281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:24.004017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1264/2529] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:24.016457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9af109-343c-44b9-a3a2-85ad761319ef admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:24.018262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-8b9af109-343c-44b9-a3a2-85ad761319ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:24.077788 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9af109-343c-44b9-a3a2-85ad761319ef admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:24.081488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dd6c8-ba6c-4057-b3f4-8e481bc72e4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:24.082094 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dd6c8-ba6c-4057-b3f4-8e481bc72e4f service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:24.082374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dd6c8-ba6c-4057-b3f4-8e481bc72e4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:24.082647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dd6c8-ba6c-4057-b3f4-8e481bc72e4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:24.091617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468dd6c8-ba6c-4057-b3f4-8e481bc72e4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:24.093174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468dd6c8-ba6c-4057-b3f4-8e481bc72e4f service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:24.101923 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1266/2530] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:25.581645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843cb51-470c-4dc9-a7f5-be17b10485cb None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:25.583066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0843cb51-470c-4dc9-a7f5-be17b10485cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:25.594933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843cb51-470c-4dc9-a7f5-be17b10485cb None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:25.598466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8479b3b-8050-462b-bc29-3cd589af4f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:25.599062 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8479b3b-8050-462b-bc29-3cd589af4f71 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:25.599337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8479b3b-8050-462b-bc29-3cd589af4f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:25.599612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8479b3b-8050-462b-bc29-3cd589af4f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:25.601337 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e8479b3b-8050-462b-bc29-3cd589af4f71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:25.668048 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8479b3b-8050-462b-bc29-3cd589af4f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:25.669651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8479b3b-8050-462b-bc29-3cd589af4f71 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:25.680359 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1265/2531] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:24:34.107520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:35.675835 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=40 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:36.949546 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dd6c8-ba6c-4057-b3f4-8e481bc72e4f service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:36.950978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-468dd6c8-ba6c-4057-b3f4-8e481bc72e4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:37.019984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dd6c8-ba6c-4057-b3f4-8e481bc72e4f service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:37.025124 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:37.025877 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:37.026152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:37.026420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] PATH_INFO: `/v3/users/bb5859187ff945a99360ae6fd0cfd5b1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:37.027589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb5859187ff945a99360ae6fd0cfd5b1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:37.029582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:37.235135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb5859187ff945a99360ae6fd0cfd5b1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:37.235782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb5859187ff945a99360ae6fd0cfd5b1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:37.242856 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb5859187ff945a99360ae6fd0cfd5b1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:37.243488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb5859187ff945a99360ae6fd0cfd5b1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:37.318661 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb5859187ff945a99360ae6fd0cfd5b1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:37.337427 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1267/2532] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:36 2022] DELETE /identity/v3/users/bb5859187ff945a99360ae6fd0cfd5b1 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:24:37.342165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a28012-d592-4c24-a0a1-9306dd051e64 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:37.342614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a28012-d592-4c24-a0a1-9306dd051e64 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:37.342726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a28012-d592-4c24-a0a1-9306dd051e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:37.367518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-54a28012-d592-4c24-a0a1-9306dd051e64 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:37.368842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-54a28012-d592-4c24-a0a1-9306dd051e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:37.442183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1266/2533] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:37.452069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:37.454134 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:37.517250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351527f-81ae-461c-b04b-37b94f2267d1 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:37.521950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1001034-c549-45f8-9d24-6bf609aa0851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:37.521950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1001034-c549-45f8-9d24-6bf609aa0851 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:37.522294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1001034-c549-45f8-9d24-6bf609aa0851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:37.522294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1001034-c549-45f8-9d24-6bf609aa0851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:37.535233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1001034-c549-45f8-9d24-6bf609aa0851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:37.537005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1001034-c549-45f8-9d24-6bf609aa0851 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:37.548689 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1268/2534] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:38.087674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a28012-d592-4c24-a0a1-9306dd051e64 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:38.089419 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-54a28012-d592-4c24-a0a1-9306dd051e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:38.102041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a28012-d592-4c24-a0a1-9306dd051e64 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:38.107199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:38.107837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:38.108122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:38.108402 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] PATH_INFO: `/v3/projects/d021de128efb4a7b8ddc9b500edb0eb3` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:38.110008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d021de128efb4a7b8ddc9b500edb0eb3)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:38.112247 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:38.274743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd021de128efb4a7b8ddc9b500edb0eb3', 'request_id': 'req-e7f2a33e-4d81-4a4d-abca-4e1acd415571'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:38.279978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd021de128efb4a7b8ddc9b500edb0eb3', 'request_id': 'req-e7f2a33e-4d81-4a4d-abca-4e1acd415571'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:38.364113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd021de128efb4a7b8ddc9b500edb0eb3', 'request_id': 'req-e7f2a33e-4d81-4a4d-abca-4e1acd415571'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:38.376478 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] The token cache is being invalidate because project d021de128efb4a7b8ddc9b500edb0eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:38.376830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:38.377528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:38.381014 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1267/2535] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:38 2022] DELETE /identity/v3/projects/d021de128efb4a7b8ddc9b500edb0eb3 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:38.383987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1001034-c549-45f8-9d24-6bf609aa0851 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:38.393576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1001034-c549-45f8-9d24-6bf609aa0851 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:38.412314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:38.451731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501e1d0-8a4a-4d94-a18a-c0ea25140c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:38.452378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501e1d0-8a4a-4d94-a18a-c0ea25140c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:38.452749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501e1d0-8a4a-4d94-a18a-c0ea25140c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:38.453293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501e1d0-8a4a-4d94-a18a-c0ea25140c98 admin admin] PATH_INFO: `/v3/users/72781b5ab4674782b18f1d78576fb6b1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:38.454579 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501e1d0-8a4a-4d94-a18a-c0ea25140c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72781b5ab4674782b18f1d78576fb6b1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:38.456522 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501e1d0-8a4a-4d94-a18a-c0ea25140c98 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:38.703580 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6501e1d0-8a4a-4d94-a18a-c0ea25140c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72781b5ab4674782b18f1d78576fb6b1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:38.704116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6501e1d0-8a4a-4d94-a18a-c0ea25140c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72781b5ab4674782b18f1d78576fb6b1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:38.711315 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6501e1d0-8a4a-4d94-a18a-c0ea25140c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72781b5ab4674782b18f1d78576fb6b1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:38.711969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6501e1d0-8a4a-4d94-a18a-c0ea25140c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72781b5ab4674782b18f1d78576fb6b1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:38.810016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-6501e1d0-8a4a-4d94-a18a-c0ea25140c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72781b5ab4674782b18f1d78576fb6b1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:38.830443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1269/2536] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:38 2022] DELETE /identity/v3/users/72781b5ab4674782b18f1d78576fb6b1 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:39.298470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:39.301312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:39.317687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2a33e-4d81-4a4d-abca-4e1acd415571 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:39.379438 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:39.380080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:39.380364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:39.380639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] PATH_INFO: `/v3/projects/b88c4f600bf74d46a0cf6e655ead0658` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:39.381790 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b88c4f600bf74d46a0cf6e655ead0658)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:39.383707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:39.497643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:39.524359 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b88c4f600bf74d46a0cf6e655ead0658', 'request_id': 'req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:39.530124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b88c4f600bf74d46a0cf6e655ead0658', 'request_id': 'req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:39.615339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b88c4f600bf74d46a0cf6e655ead0658', 'request_id': 'req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:39.627981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] The token cache is being invalidate because project b88c4f600bf74d46a0cf6e655ead0658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:39.628274 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:39.628932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c5cc5bde-2a54-4d3e-bf43-7da3ed2f055c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:39.630499 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1268/2537] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:39 2022] DELETE /identity/v3/projects/b88c4f600bf74d46a0cf6e655ead0658 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:39.635473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824ba58-54a0-4182-af1c-8cc0bfe8e8db None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:39.635953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824ba58-54a0-4182-af1c-8cc0bfe8e8db None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:39.636077 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824ba58-54a0-4182-af1c-8cc0bfe8e8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:39.655670 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-2824ba58-54a0-4182-af1c-8cc0bfe8e8db None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:39.657891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2824ba58-54a0-4182-af1c-8cc0bfe8e8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:39.715385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1270/2538] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:39.725369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5393cf4-07ee-4b8c-9bf5-ac14cf7adca6 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:39.725657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5393cf4-07ee-4b8c-9bf5-ac14cf7adca6 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:39.725934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5393cf4-07ee-4b8c-9bf5-ac14cf7adca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:39.746592 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-a5393cf4-07ee-4b8c-9bf5-ac14cf7adca6 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:39.747838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a5393cf4-07ee-4b8c-9bf5-ac14cf7adca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:39.755799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1269/2539] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:39.760054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824ba58-54a0-4182-af1c-8cc0bfe8e8db None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:39.770070 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824ba58-54a0-4182-af1c-8cc0bfe8e8db None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:39.775400 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8974aa1-70aa-41cb-b5d9-2fe8d9b8cfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:39.775998 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8974aa1-70aa-41cb-b5d9-2fe8d9b8cfbc admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:39.776279 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8974aa1-70aa-41cb-b5d9-2fe8d9b8cfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:39.776563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8974aa1-70aa-41cb-b5d9-2fe8d9b8cfbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:39.777233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8974aa1-70aa-41cb-b5d9-2fe8d9b8cfbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:24:39.777542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8974aa1-70aa-41cb-b5d9-2fe8d9b8cfbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:39.779154 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8974aa1-70aa-41cb-b5d9-2fe8d9b8cfbc admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:39.792761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1271/2540] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:24:39.797850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5393cf4-07ee-4b8c-9bf5-ac14cf7adca6 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:39.807285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5393cf4-07ee-4b8c-9bf5-ac14cf7adca6 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:39.811939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961a60b-caf6-4e1f-ae80-10e64b543adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:39.812526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961a60b-caf6-4e1f-ae80-10e64b543adc admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:39.812800 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961a60b-caf6-4e1f-ae80-10e64b543adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:39.813100 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961a60b-caf6-4e1f-ae80-10e64b543adc admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:39.813724 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961a60b-caf6-4e1f-ae80-10e64b543adc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:39.815605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961a60b-caf6-4e1f-ae80-10e64b543adc admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:39.896399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1270/2541] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:24:39 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:39.900637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8974aa1-70aa-41cb-b5d9-2fe8d9b8cfbc admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:39.911708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8974aa1-70aa-41cb-b5d9-2fe8d9b8cfbc admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:39.939653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:39.969694 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb7881-0f65-4c59-96b6-ee9c02418fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:39.970160 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb7881-0f65-4c59-96b6-ee9c02418fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:39.970438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb7881-0f65-4c59-96b6-ee9c02418fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:39.970713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb7881-0f65-4c59-96b6-ee9c02418fa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:39.971324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb7881-0f65-4c59-96b6-ee9c02418fa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:39.973261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb7881-0f65-4c59-96b6-ee9c02418fa5 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:40.106591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1272/2542] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:24:39 2022] POST /identity/v3/users => generated 284 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:40.111421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961a60b-caf6-4e1f-ae80-10e64b543adc admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:40.121934 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961a60b-caf6-4e1f-ae80-10e64b543adc admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:40.126629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da028ddb-0f65-46e2-a983-dc9e336bab02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:40.127230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da028ddb-0f65-46e2-a983-dc9e336bab02 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:40.127510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da028ddb-0f65-46e2-a983-dc9e336bab02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:40.127800 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da028ddb-0f65-46e2-a983-dc9e336bab02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:40.128460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da028ddb-0f65-46e2-a983-dc9e336bab02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:40.130094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da028ddb-0f65-46e2-a983-dc9e336bab02 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:40.140474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1271/2543] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:40.144743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb7881-0f65-4c59-96b6-ee9c02418fa5 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:40.155086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb7881-0f65-4c59-96b6-ee9c02418fa5 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:40.162340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfd9fa-fb1d-442a-8fa4-eb0ff3df7116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:40.162976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfd9fa-fb1d-442a-8fa4-eb0ff3df7116 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:40.163407 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfd9fa-fb1d-442a-8fa4-eb0ff3df7116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:40.163509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfd9fa-fb1d-442a-8fa4-eb0ff3df7116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:40.164146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dfd9fa-fb1d-442a-8fa4-eb0ff3df7116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:40.165789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dfd9fa-fb1d-442a-8fa4-eb0ff3df7116 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:40.176914 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1273/2544] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:40.181116 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da028ddb-0f65-46e2-a983-dc9e336bab02 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:40.191126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da028ddb-0f65-46e2-a983-dc9e336bab02 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:40.195666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581ca14-34d1-4956-92fd-b2f01f8a39d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:40.196314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581ca14-34d1-4956-92fd-b2f01f8a39d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:40.196593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581ca14-34d1-4956-92fd-b2f01f8a39d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:40.196888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581ca14-34d1-4956-92fd-b2f01f8a39d1 admin admin] PATH_INFO: `/v3/projects/5490a446b67f421f9f244d7d3b283dde/users/0ad96e20bd4e4a799e135f298aaaf372/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:40.211920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581ca14-34d1-4956-92fd-b2f01f8a39d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5490a446b67f421f9f244d7d3b283dde, user_id=0ad96e20bd4e4a799e135f298aaaf372, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:40.214659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581ca14-34d1-4956-92fd-b2f01f8a39d1 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:40.319489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1272/2545] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:40 2022] PUT /identity/v3/projects/5490a446b67f421f9f244d7d3b283dde/users/0ad96e20bd4e4a799e135f298aaaf372/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:40.330611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8411789-fc8f-4108-9546-e834da60d080 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:40.330901 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8411789-fc8f-4108-9546-e834da60d080 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:40.331178 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8411789-fc8f-4108-9546-e834da60d080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:40.362127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e8411789-fc8f-4108-9546-e834da60d080 None None] MFA Rules not processed for user `0ad96e20bd4e4a799e135f298aaaf372`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:40.363387 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e8411789-fc8f-4108-9546-e834da60d080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:40.427900 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1274/2546] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:40.432432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581ca14-34d1-4956-92fd-b2f01f8a39d1 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:40.496202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581ca14-34d1-4956-92fd-b2f01f8a39d1 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:40.501477 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864c229-7ad9-451b-bb87-a3274239ecf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:40.502082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864c229-7ad9-451b-bb87-a3274239ecf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:40.502362 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864c229-7ad9-451b-bb87-a3274239ecf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:40.502665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864c229-7ad9-451b-bb87-a3274239ecf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:40.503290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864c229-7ad9-451b-bb87-a3274239ecf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:40.505258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864c229-7ad9-451b-bb87-a3274239ecf9 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:40.596719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1273/2547] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:24:40 2022] POST /identity/v3/projects => generated 367 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:40.597865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8411789-fc8f-4108-9546-e834da60d080 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:40.608735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8411789-fc8f-4108-9546-e834da60d080 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:40.664199 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbbc412-6b61-4195-8550-a66387b59484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:40.664903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbc412-6b61-4195-8550-a66387b59484 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:40.665240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbc412-6b61-4195-8550-a66387b59484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:40.665608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbc412-6b61-4195-8550-a66387b59484 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:40.666258 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbbc412-6b61-4195-8550-a66387b59484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:40.668506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbbc412-6b61-4195-8550-a66387b59484 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:40.781764 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1275/2548] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:24:40 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:40.785768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864c229-7ad9-451b-bb87-a3274239ecf9 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:40.796746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864c229-7ad9-451b-bb87-a3274239ecf9 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:40.801681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47173867-563b-4a2e-b9f7-e05cbbf3a4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:40.802285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47173867-563b-4a2e-b9f7-e05cbbf3a4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:40.802566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47173867-563b-4a2e-b9f7-e05cbbf3a4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:40.802842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47173867-563b-4a2e-b9f7-e05cbbf3a4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:40.803485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47173867-563b-4a2e-b9f7-e05cbbf3a4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:40.805122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47173867-563b-4a2e-b9f7-e05cbbf3a4c5 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:40.816366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1274/2549] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:40.820455 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbbc412-6b61-4195-8550-a66387b59484 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:40.830698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbbc412-6b61-4195-8550-a66387b59484 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:40.835830 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876d9d7-ff04-4570-95b2-e7b96e732803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:40.836472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876d9d7-ff04-4570-95b2-e7b96e732803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:40.836750 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876d9d7-ff04-4570-95b2-e7b96e732803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:40.837058 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876d9d7-ff04-4570-95b2-e7b96e732803 admin admin] PATH_INFO: `/v3/projects/10d42090eb8e43b889ef40c00872c7a0/users/3e9c374fca174fbcacb1c1cfba2149c4/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:40.852555 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876d9d7-ff04-4570-95b2-e7b96e732803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d42090eb8e43b889ef40c00872c7a0, user_id=3e9c374fca174fbcacb1c1cfba2149c4, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:40.855310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876d9d7-ff04-4570-95b2-e7b96e732803 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:40.929330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1276/2550] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:40 2022] PUT /identity/v3/projects/10d42090eb8e43b889ef40c00872c7a0/users/3e9c374fca174fbcacb1c1cfba2149c4/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:40.933486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47173867-563b-4a2e-b9f7-e05cbbf3a4c5 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:40.944270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47173867-563b-4a2e-b9f7-e05cbbf3a4c5 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:40.964069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:41.004968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccee14-9cb7-4b0a-8728-f9a661d5c86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:41.005586 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccee14-9cb7-4b0a-8728-f9a661d5c86e admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:41.005872 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccee14-9cb7-4b0a-8728-f9a661d5c86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:41.006177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccee14-9cb7-4b0a-8728-f9a661d5c86e admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:41.006818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccee14-9cb7-4b0a-8728-f9a661d5c86e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:41.008454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccee14-9cb7-4b0a-8728-f9a661d5c86e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:41.019277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1275/2551] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:24:41.025762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876d9d7-ff04-4570-95b2-e7b96e732803 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:41.032336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:41.037828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876d9d7-ff04-4570-95b2-e7b96e732803 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:41.042556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577c3a3-b7af-4980-af7d-d806d9856a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:41.043232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577c3a3-b7af-4980-af7d-d806d9856a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:41.043512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577c3a3-b7af-4980-af7d-d806d9856a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:41.043789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577c3a3-b7af-4980-af7d-d806d9856a03 admin admin] PATH_INFO: `/v3/projects/10d42090eb8e43b889ef40c00872c7a0/users/3e9c374fca174fbcacb1c1cfba2149c4/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:41.045749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577c3a3-b7af-4980-af7d-d806d9856a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d42090eb8e43b889ef40c00872c7a0, user_id=3e9c374fca174fbcacb1c1cfba2149c4, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:41.048993 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577c3a3-b7af-4980-af7d-d806d9856a03 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:41.059270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1277/2552] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:41 2022] PUT /identity/v3/projects/10d42090eb8e43b889ef40c00872c7a0/users/3e9c374fca174fbcacb1c1cfba2149c4/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:41.070424 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc6fce-fea3-4aac-91db-4cb268195d45 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:41.070715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc6fce-fea3-4aac-91db-4cb268195d45 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:41.071126 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc6fce-fea3-4aac-91db-4cb268195d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:41.103352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-77bc6fce-fea3-4aac-91db-4cb268195d45 None None] MFA Rules not processed for user `3e9c374fca174fbcacb1c1cfba2149c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:41.104617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-77bc6fce-fea3-4aac-91db-4cb268195d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:41.176098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1276/2553] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:41.187952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577c3a3-b7af-4980-af7d-d806d9856a03 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:41.189682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b577c3a3-b7af-4980-af7d-d806d9856a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:41.252368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577c3a3-b7af-4980-af7d-d806d9856a03 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:41.255986 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0cb4d-72e7-477f-9079-f841bb7577a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:41.256592 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0cb4d-72e7-477f-9079-f841bb7577a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:41.256887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0cb4d-72e7-477f-9079-f841bb7577a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:41.257180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0cb4d-72e7-477f-9079-f841bb7577a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:41.266647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0cb4d-72e7-477f-9079-f841bb7577a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:41.268095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0cb4d-72e7-477f-9079-f841bb7577a4 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:41.278466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1278/2554] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:41.510156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc6fce-fea3-4aac-91db-4cb268195d45 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:41.522792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc6fce-fea3-4aac-91db-4cb268195d45 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:41.528486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b28286-79db-4ded-a504-b3587148982c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:41.529509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28286-79db-4ded-a504-b3587148982c service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:41.529992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28286-79db-4ded-a504-b3587148982c service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:41.531012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28286-79db-4ded-a504-b3587148982c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:41.616427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b28286-79db-4ded-a504-b3587148982c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:41.617922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b28286-79db-4ded-a504-b3587148982c service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:41.628186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1277/2555] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:48.865034 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0cb4d-72e7-477f-9079-f841bb7577a4 service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:48.991744 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0cb4d-72e7-477f-9079-f841bb7577a4 service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:49.001284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:49.002493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:49.002849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:49.003137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] PATH_INFO: `/v3/users/0ad96e20bd4e4a799e135f298aaaf372` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:49.004358 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ad96e20bd4e4a799e135f298aaaf372)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:49.007275 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:49.233418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ad96e20bd4e4a799e135f298aaaf372'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:49.234477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ad96e20bd4e4a799e135f298aaaf372'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:49.244005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ad96e20bd4e4a799e135f298aaaf372'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:49.244744 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ad96e20bd4e4a799e135f298aaaf372'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:49.329781 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ad96e20bd4e4a799e135f298aaaf372'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:49.350676 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1279/2556] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:48 2022] DELETE /identity/v3/users/0ad96e20bd4e4a799e135f298aaaf372 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:49.355681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429f930-2dfc-4b4f-9e5b-21114f82d652 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:49.356302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429f930-2dfc-4b4f-9e5b-21114f82d652 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:49.356400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429f930-2dfc-4b4f-9e5b-21114f82d652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:49.380508 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b429f930-2dfc-4b4f-9e5b-21114f82d652 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:49.381943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b429f930-2dfc-4b4f-9e5b-21114f82d652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:49.428754 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:49.456777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1278/2557] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:49.472306 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:49.473896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:49.490824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:49.543484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a5e2e-a0bb-45fb-8518-2c22ceaadb61 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:49.547997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd8a25-56bc-42e6-a189-d65e7eef8b6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:49.548752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd8a25-56bc-42e6-a189-d65e7eef8b6d service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:49.549311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd8a25-56bc-42e6-a189-d65e7eef8b6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:49.549599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd8a25-56bc-42e6-a189-d65e7eef8b6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:49.561121 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd8a25-56bc-42e6-a189-d65e7eef8b6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:49.563520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd8a25-56bc-42e6-a189-d65e7eef8b6d service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:49.575212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1280/2558] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:49.666787 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429f930-2dfc-4b4f-9e5b-21114f82d652 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:49.668381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b429f930-2dfc-4b4f-9e5b-21114f82d652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:49.683255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429f930-2dfc-4b4f-9e5b-21114f82d652 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:49.688685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:49.689412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:49.695850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:49.695850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] PATH_INFO: `/v3/users/293fbfbe348f4b73ad3dc9c0633d9c74` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:49.695850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=293fbfbe348f4b73ad3dc9c0633d9c74)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:49.695850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:49.932827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '293fbfbe348f4b73ad3dc9c0633d9c74'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:49.933515 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '293fbfbe348f4b73ad3dc9c0633d9c74'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:49.949195 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '293fbfbe348f4b73ad3dc9c0633d9c74'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:49.970985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '293fbfbe348f4b73ad3dc9c0633d9c74'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.074035 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd8a25-56bc-42e6-a189-d65e7eef8b6d service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:50.075972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-71fd8a25-56bc-42e6-a189-d65e7eef8b6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:50.089721 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd8a25-56bc-42e6-a189-d65e7eef8b6d service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:50.094494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:50.099707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:50.099707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:50.099707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] PATH_INFO: `/v3/projects/5490a446b67f421f9f244d7d3b283dde` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:50.099707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5490a446b67f421f9f244d7d3b283dde)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:50.099707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:50.129548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '293fbfbe348f4b73ad3dc9c0633d9c74'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.135906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1279/2559] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:49 2022] DELETE /identity/v3/users/293fbfbe348f4b73ad3dc9c0633d9c74 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:24:50.208303 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5490a446b67f421f9f244d7d3b283dde', 'request_id': 'req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.215590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5490a446b67f421f9f244d7d3b283dde', 'request_id': 'req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.223314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5490a446b67f421f9f244d7d3b283dde', 'request_id': 'req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.296541 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] The token cache is being invalidate because project 5490a446b67f421f9f244d7d3b283dde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:50.296840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.297664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.299175 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1281/2560] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:50 2022] DELETE /identity/v3/projects/5490a446b67f421f9f244d7d3b283dde => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:50.303330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:50.304902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:50.373122 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b757c-6aae-4c94-a0e8-f0b13a282019 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:50.431258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:50.431881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:50.432151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:50.432427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] PATH_INFO: `/v3/users/3e9c374fca174fbcacb1c1cfba2149c4` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:50.433619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e9c374fca174fbcacb1c1cfba2149c4)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:50.435531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:50.440225 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:50.509182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:50.510911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:50.526393 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff08ab-f8db-4c33-9000-24d1dd7d0ba7 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:50.531572 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:50.532215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:50.535115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:50.535115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] PATH_INFO: `/v3/projects/3456d6c9778140e486b78e739a0588e2` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:50.535115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3456d6c9778140e486b78e739a0588e2)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:50.535829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:50.624548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e9c374fca174fbcacb1c1cfba2149c4'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.625350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e9c374fca174fbcacb1c1cfba2149c4'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.632415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e9c374fca174fbcacb1c1cfba2149c4'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.645536 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e9c374fca174fbcacb1c1cfba2149c4'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.684500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3456d6c9778140e486b78e739a0588e2', 'request_id': 'req-ddae33e0-99c1-40cb-8631-55f61820b5ba'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.690638 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3456d6c9778140e486b78e739a0588e2', 'request_id': 'req-ddae33e0-99c1-40cb-8631-55f61820b5ba'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.697089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3456d6c9778140e486b78e739a0588e2', 'request_id': 'req-ddae33e0-99c1-40cb-8631-55f61820b5ba'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.729186 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e9c374fca174fbcacb1c1cfba2149c4'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.735859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1280/2561] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:50 2022] DELETE /identity/v3/users/3e9c374fca174fbcacb1c1cfba2149c4 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:50.816615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] The token cache is being invalidate because project 3456d6c9778140e486b78e739a0588e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:50.817006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.817839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:50.819307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1282/2562] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:50 2022] DELETE /identity/v3/projects/3456d6c9778140e486b78e739a0588e2 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:24:50.823341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:50.824989 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:50.902948 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a06d99-ea56-4924-891f-11c78ac84a47 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:50.908770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:50.909665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:50.909939 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:50.910208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] PATH_INFO: `/v3/users/128aa8503268455d8a735b5129362b40` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:50.911436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=128aa8503268455d8a735b5129362b40)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:50.913392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:51.143977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128aa8503268455d8a735b5129362b40'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:51.144628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128aa8503268455d8a735b5129362b40'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:51.151588 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128aa8503268455d8a735b5129362b40'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:51.173740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128aa8503268455d8a735b5129362b40'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:51.260824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128aa8503268455d8a735b5129362b40'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:51.268290 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1281/2563] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:50 2022] DELETE /identity/v3/users/128aa8503268455d8a735b5129362b40 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:24:51.306302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:51.308772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:51.348549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:51.416033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae33e0-99c1-40cb-8631-55f61820b5ba admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:51.491854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:51.493693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:51.493693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:51.493693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] PATH_INFO: `/v3/projects/10d42090eb8e43b889ef40c00872c7a0` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:51.494664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d42090eb8e43b889ef40c00872c7a0)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:51.496588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:51.605724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d42090eb8e43b889ef40c00872c7a0', 'request_id': 'req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:51.614364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d42090eb8e43b889ef40c00872c7a0', 'request_id': 'req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:51.621152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d42090eb8e43b889ef40c00872c7a0', 'request_id': 'req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:51.708094 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:51.712015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:51.719571 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] The token cache is being invalidate because project 10d42090eb8e43b889ef40c00872c7a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:51.720112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:51.721097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-049cd4e8-ec76-4d1e-b346-59d0cba15ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:51.723119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1283/2564] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:51 2022] DELETE /identity/v3/projects/10d42090eb8e43b889ef40c00872c7a0 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:51.723550 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:51.728355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49882408-829b-422a-b7db-b389e3b2eec1 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:51.728642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49882408-829b-422a-b7db-b389e3b2eec1 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:51.728945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49882408-829b-422a-b7db-b389e3b2eec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:51.734717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be4d59-72ab-46a0-a866-5d94d6c2ae04 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:51.750724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-49882408-829b-422a-b7db-b389e3b2eec1 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:51.752027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-49882408-829b-422a-b7db-b389e3b2eec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:51.820652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:51.822111 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:51.822494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:51.822975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] PATH_INFO: `/v3/projects/4d309a4f701e40b197f5650df92c703b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:51.824149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:51.825263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d309a4f701e40b197f5650df92c703b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:51.827824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:51.834616 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1284/2565] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:51.843588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d377d8-53b6-40ce-8ff4-1ed4ca835f77 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:51.843880 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d377d8-53b6-40ce-8ff4-1ed4ca835f77 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:51.844155 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d377d8-53b6-40ce-8ff4-1ed4ca835f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:51.868132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-94d377d8-53b6-40ce-8ff4-1ed4ca835f77 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:51.869405 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-94d377d8-53b6-40ce-8ff4-1ed4ca835f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:51.878386 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1285/2566] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:24:51.882589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d377d8-53b6-40ce-8ff4-1ed4ca835f77 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:51.892037 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d377d8-53b6-40ce-8ff4-1ed4ca835f77 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:51.897820 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8a0fb-96cc-4a83-a6d2-9b1e8f38d268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:51.900111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8a0fb-96cc-4a83-a6d2-9b1e8f38d268 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:51.900111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8a0fb-96cc-4a83-a6d2-9b1e8f38d268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:51.900111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8a0fb-96cc-4a83-a6d2-9b1e8f38d268 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:51.900111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8a0fb-96cc-4a83-a6d2-9b1e8f38d268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:24:51.900111 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8a0fb-96cc-4a83-a6d2-9b1e8f38d268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:51.901559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8a0fb-96cc-4a83-a6d2-9b1e8f38d268 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:51.923953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1286/2567] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:24:51.929237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8a0fb-96cc-4a83-a6d2-9b1e8f38d268 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:51.940306 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8a0fb-96cc-4a83-a6d2-9b1e8f38d268 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:51.945162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d866c8-3927-4b1f-a0ac-5e6c40a1bf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:51.945766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d866c8-3927-4b1f-a0ac-5e6c40a1bf41 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:51.946046 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d866c8-3927-4b1f-a0ac-5e6c40a1bf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:51.946342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d866c8-3927-4b1f-a0ac-5e6c40a1bf41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:51.946973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d866c8-3927-4b1f-a0ac-5e6c40a1bf41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:51.948876 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d866c8-3927-4b1f-a0ac-5e6c40a1bf41 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:52.004773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d309a4f701e40b197f5650df92c703b', 'request_id': 'req-d0e00beb-4262-4257-9da3-d79abfafad27'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:52.016908 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d309a4f701e40b197f5650df92c703b', 'request_id': 'req-d0e00beb-4262-4257-9da3-d79abfafad27'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:52.095330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d309a4f701e40b197f5650df92c703b', 'request_id': 'req-d0e00beb-4262-4257-9da3-d79abfafad27'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:52.106317 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1287/2568] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:24:51 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:52.110921 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] The token cache is being invalidate because project 4d309a4f701e40b197f5650df92c703b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:24:52.111404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:52.111755 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d866c8-3927-4b1f-a0ac-5e6c40a1bf41 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:52.112564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-d0e00beb-4262-4257-9da3-d79abfafad27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:52.114376 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1282/2569] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:51 2022] DELETE /identity/v3/projects/4d309a4f701e40b197f5650df92c703b => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:52.119063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59590a21-49ff-49eb-8c8d-044e0ee0be88 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.119527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59590a21-49ff-49eb-8c8d-044e0ee0be88 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.119805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59590a21-49ff-49eb-8c8d-044e0ee0be88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.142659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-59590a21-49ff-49eb-8c8d-044e0ee0be88 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:52.143999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-59590a21-49ff-49eb-8c8d-044e0ee0be88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:52.209985 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d866c8-3927-4b1f-a0ac-5e6c40a1bf41 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:52.215777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738271d-a4da-485c-83d8-f886a793012f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:52.216771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738271d-a4da-485c-83d8-f886a793012f admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.216771 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738271d-a4da-485c-83d8-f886a793012f admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.217020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738271d-a4da-485c-83d8-f886a793012f admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.218546 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738271d-a4da-485c-83d8-f886a793012f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:52.226839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738271d-a4da-485c-83d8-f886a793012f admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:52.269253 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1283/2570] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:24:52.273731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c206e-5eb8-4577-911b-7efd45dd9829 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.274021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c206e-5eb8-4577-911b-7efd45dd9829 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.274300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c206e-5eb8-4577-911b-7efd45dd9829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.298114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-cf3c206e-5eb8-4577-911b-7efd45dd9829 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:52.299534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-cf3c206e-5eb8-4577-911b-7efd45dd9829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:52.307589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1284/2571] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:24:52.312536 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c206e-5eb8-4577-911b-7efd45dd9829 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:52.322041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c206e-5eb8-4577-911b-7efd45dd9829 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:52.328715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac92f95-182b-4c90-88dc-4fd716c15b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:52.329367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac92f95-182b-4c90-88dc-4fd716c15b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.329647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac92f95-182b-4c90-88dc-4fd716c15b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.329918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac92f95-182b-4c90-88dc-4fd716c15b55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.330578 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac92f95-182b-4c90-88dc-4fd716c15b55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:24:52.330874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac92f95-182b-4c90-88dc-4fd716c15b55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:52.334109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1288/2572] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:24:52 2022] POST /identity/v3/users => generated 284 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:24:52.334741 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac92f95-182b-4c90-88dc-4fd716c15b55 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:52.342421 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738271d-a4da-485c-83d8-f886a793012f admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:52.342421 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e738271d-a4da-485c-83d8-f886a793012f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:52.353002 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738271d-a4da-485c-83d8-f886a793012f admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:52.357044 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1285/2573] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:24:52.358255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b52da-728c-4e69-b6ab-11e85d079866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:52.359320 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b52da-728c-4e69-b6ab-11e85d079866 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.359630 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b52da-728c-4e69-b6ab-11e85d079866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.359904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b52da-728c-4e69-b6ab-11e85d079866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.360556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b52da-728c-4e69-b6ab-11e85d079866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:52.362521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b52da-728c-4e69-b6ab-11e85d079866 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:52.362691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac92f95-182b-4c90-88dc-4fd716c15b55 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:52.375199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac92f95-182b-4c90-88dc-4fd716c15b55 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:52.379598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1289/2574] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:24:52.380603 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b52da-728c-4e69-b6ab-11e85d079866 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:52.384732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae195c2c-93dd-4b3a-8635-528d363dcdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:52.385516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae195c2c-93dd-4b3a-8635-528d363dcdae admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.386219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae195c2c-93dd-4b3a-8635-528d363dcdae admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.386537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae195c2c-93dd-4b3a-8635-528d363dcdae admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.388091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae195c2c-93dd-4b3a-8635-528d363dcdae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:52.391590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae195c2c-93dd-4b3a-8635-528d363dcdae admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:52.392340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b52da-728c-4e69-b6ab-11e85d079866 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:52.397196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37c1b2-261b-4b6a-9f59-c2010624d1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:52.397961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37c1b2-261b-4b6a-9f59-c2010624d1de admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.398241 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37c1b2-261b-4b6a-9f59-c2010624d1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.398537 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37c1b2-261b-4b6a-9f59-c2010624d1de admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.399161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37c1b2-261b-4b6a-9f59-c2010624d1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:52.400757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37c1b2-261b-4b6a-9f59-c2010624d1de admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:52.408338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:52.416894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1290/2575] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:24:52.422221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37c1b2-261b-4b6a-9f59-c2010624d1de admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:52.433553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37c1b2-261b-4b6a-9f59-c2010624d1de admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:52.439815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b6ba5-c40a-4f84-9c5a-f352a4f73cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:52.440491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b6ba5-c40a-4f84-9c5a-f352a4f73cae admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.440779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b6ba5-c40a-4f84-9c5a-f352a4f73cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.441097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b6ba5-c40a-4f84-9c5a-f352a4f73cae admin admin] PATH_INFO: `/v3/projects/442594393ab4477bbb588b9e25186998/users/101064af8a684b6782f1ebfca51db0eb/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.458089 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b6ba5-c40a-4f84-9c5a-f352a4f73cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=442594393ab4477bbb588b9e25186998, user_id=101064af8a684b6782f1ebfca51db0eb, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:52.461269 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b6ba5-c40a-4f84-9c5a-f352a4f73cae admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:52.505865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1286/2576] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:24:52 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:24:52.507828 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae195c2c-93dd-4b3a-8635-528d363dcdae admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:52.560339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1291/2577] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:52 2022] PUT /identity/v3/projects/442594393ab4477bbb588b9e25186998/users/101064af8a684b6782f1ebfca51db0eb/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:24:52.574854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd118-e793-4aab-ba24-529d985e8d2d None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.576065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd118-e793-4aab-ba24-529d985e8d2d None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.578688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd118-e793-4aab-ba24-529d985e8d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.636689 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c3bbd118-e793-4aab-ba24-529d985e8d2d None None] MFA Rules not processed for user `101064af8a684b6782f1ebfca51db0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:52.638628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c3bbd118-e793-4aab-ba24-529d985e8d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:52.672492 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae195c2c-93dd-4b3a-8635-528d363dcdae admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:52.726682 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:52.774338 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1292/2578] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:52.789117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b01b2-1480-455e-840e-63f9affee7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:52.789117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b01b2-1480-455e-840e-63f9affee7fb admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.789117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b01b2-1480-455e-840e-63f9affee7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.789117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b01b2-1480-455e-840e-63f9affee7fb admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.790558 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b01b2-1480-455e-840e-63f9affee7fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:52.790558 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b01b2-1480-455e-840e-63f9affee7fb admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:52.813595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbd118-e793-4aab-ba24-529d985e8d2d None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:52.815858 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c3bbd118-e793-4aab-ba24-529d985e8d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:52.895545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbd118-e793-4aab-ba24-529d985e8d2d None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:52.899852 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5e4ce-b44a-4310-ae52-d8000aaa1e54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0b124ca267b4a4391557ad1396524cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:52.900680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e4ce-b44a-4310-ae52-d8000aaa1e54 service glance] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.901074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e4ce-b44a-4310-ae52-d8000aaa1e54 service glance] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.901372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e4ce-b44a-4310-ae52-d8000aaa1e54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.911139 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5e4ce-b44a-4310-ae52-d8000aaa1e54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:52.912593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5e4ce-b44a-4310-ae52-d8000aaa1e54 service glance] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:52.922141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1293/2579] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:52.956039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1287/2580] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:24:52 2022] POST /identity/v3/users => generated 288 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:24:52.962625 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5e4ce-b44a-4310-ae52-d8000aaa1e54 service glance] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:52.977303 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5e4ce-b44a-4310-ae52-d8000aaa1e54 service glance] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:52.982312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35938840-b82b-43c1-aa2e-cffd2451f624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:52.983068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35938840-b82b-43c1-aa2e-cffd2451f624 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:52.983480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35938840-b82b-43c1-aa2e-cffd2451f624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:52.984010 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35938840-b82b-43c1-aa2e-cffd2451f624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:52.984886 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35938840-b82b-43c1-aa2e-cffd2451f624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:52.986607 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35938840-b82b-43c1-aa2e-cffd2451f624 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:53.001487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1294/2581] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:24:53.002705 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b01b2-1480-455e-840e-63f9affee7fb admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:53.014848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b01b2-1480-455e-840e-63f9affee7fb admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:53.020129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c1bd4-12bc-4ebe-96af-75c7371d728b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:53.027971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c1bd4-12bc-4ebe-96af-75c7371d728b admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:53.027971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c1bd4-12bc-4ebe-96af-75c7371d728b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:53.027971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c1bd4-12bc-4ebe-96af-75c7371d728b admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:53.027971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c1bd4-12bc-4ebe-96af-75c7371d728b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:53.027971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c1bd4-12bc-4ebe-96af-75c7371d728b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:53.046648 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1288/2582] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:24:53.050116 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35938840-b82b-43c1-aa2e-cffd2451f624 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:53.076724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35938840-b82b-43c1-aa2e-cffd2451f624 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:53.087142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f3fec-944c-42e8-ae98-4df11165ce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:53.088181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f3fec-944c-42e8-ae98-4df11165ce5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:53.091132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f3fec-944c-42e8-ae98-4df11165ce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:53.091132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f3fec-944c-42e8-ae98-4df11165ce5a admin admin] PATH_INFO: `/v3/projects/6fac524cc46d4271b8f63e4a847af69c/users/df9d9de37cfb4e31bb1864d92fe96e15/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:53.110260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f3fec-944c-42e8-ae98-4df11165ce5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fac524cc46d4271b8f63e4a847af69c, user_id=df9d9de37cfb4e31bb1864d92fe96e15, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:53.113287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f3fec-944c-42e8-ae98-4df11165ce5a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:53.278171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1295/2583] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:24:53 2022] PUT /identity/v3/projects/6fac524cc46d4271b8f63e4a847af69c/users/df9d9de37cfb4e31bb1864d92fe96e15/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:53.300707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30213fc6-ed73-4e26-ad9a-0bb5560cc446 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:53.300707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30213fc6-ed73-4e26-ad9a-0bb5560cc446 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:53.301070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30213fc6-ed73-4e26-ad9a-0bb5560cc446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:53.365029 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-30213fc6-ed73-4e26-ad9a-0bb5560cc446 None None] MFA Rules not processed for user `df9d9de37cfb4e31bb1864d92fe96e15`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:53.366338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-30213fc6-ed73-4e26-ad9a-0bb5560cc446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:53.474738 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1289/2584] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:24:53.490359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f3fec-944c-42e8-ae98-4df11165ce5a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:53.492548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d83f3fec-944c-42e8-ae98-4df11165ce5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:53.569615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f3fec-944c-42e8-ae98-4df11165ce5a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:53.573291 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a466dfc-bc18-4f18-bd3c-62301779df96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:53.573912 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466dfc-bc18-4f18-bd3c-62301779df96 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:53.574190 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466dfc-bc18-4f18-bd3c-62301779df96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:53.574467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466dfc-bc18-4f18-bd3c-62301779df96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:53.585177 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a466dfc-bc18-4f18-bd3c-62301779df96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:53.587023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a466dfc-bc18-4f18-bd3c-62301779df96 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:53.599120 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1296/2585] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:58.223742 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30213fc6-ed73-4e26-ad9a-0bb5560cc446 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:58.234223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:58.345209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30213fc6-ed73-4e26-ad9a-0bb5560cc446 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:58.351493 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:58.352256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:58.352568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:58.352845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] PATH_INFO: `/v3/users/df9d9de37cfb4e31bb1864d92fe96e15` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:58.354128 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df9d9de37cfb4e31bb1864d92fe96e15)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:58.356078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:58.449256 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a466dfc-bc18-4f18-bd3c-62301779df96 service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:58.452615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2a466dfc-bc18-4f18-bd3c-62301779df96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:58.525325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:58.548966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a466dfc-bc18-4f18-bd3c-62301779df96 service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:58.552910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc21ea-f9ee-4b97-a1fb-f5f566f8154c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:58.553531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc21ea-f9ee-4b97-a1fb-f5f566f8154c service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:58.553815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc21ea-f9ee-4b97-a1fb-f5f566f8154c service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:58.554091 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc21ea-f9ee-4b97-a1fb-f5f566f8154c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:58.555929 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0adc21ea-f9ee-4b97-a1fb-f5f566f8154c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:58.585746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc21ea-f9ee-4b97-a1fb-f5f566f8154c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:58.587226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc21ea-f9ee-4b97-a1fb-f5f566f8154c service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:58.604049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1297/2586] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:24:58.694220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df9d9de37cfb4e31bb1864d92fe96e15'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:58.695012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df9d9de37cfb4e31bb1864d92fe96e15'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:58.703966 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df9d9de37cfb4e31bb1864d92fe96e15'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:58.721997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df9d9de37cfb4e31bb1864d92fe96e15'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:58.808987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df9d9de37cfb4e31bb1864d92fe96e15'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:24:58.817350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1290/2587] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:24:58 2022] DELETE /identity/v3/users/df9d9de37cfb4e31bb1864d92fe96e15 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:24:58.822974 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97e6f5-55b1-426a-930e-135656871b7e None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:58.824170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97e6f5-55b1-426a-930e-135656871b7e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:58.824170 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97e6f5-55b1-426a-930e-135656871b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:58.986868 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6e97e6f5-55b1-426a-930e-135656871b7e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:58.988104 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6e97e6f5-55b1-426a-930e-135656871b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:59.055000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1298/2588] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:24:59.065817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:59.067295 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:59.117777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:59.129401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d885dc1-f63c-45e3-8063-857dda0324f2 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:59.133411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3759-87c3-4dc7-afcc-1beba4dea940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:59.134082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3759-87c3-4dc7-afcc-1beba4dea940 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:59.134480 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3759-87c3-4dc7-afcc-1beba4dea940 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:59.134953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3759-87c3-4dc7-afcc-1beba4dea940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:59.146820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a3759-87c3-4dc7-afcc-1beba4dea940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:59.148245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a3759-87c3-4dc7-afcc-1beba4dea940 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:59.161471 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1291/2589] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:24:59.777772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97e6f5-55b1-426a-930e-135656871b7e None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:59.781643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6e97e6f5-55b1-426a-930e-135656871b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:59.804381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97e6f5-55b1-426a-930e-135656871b7e None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:59.814558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:59.815225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:59.815512 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:59.815789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] PATH_INFO: `/v3/projects/6fac524cc46d4271b8f63e4a847af69c` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:59.817030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fac524cc46d4271b8f63e4a847af69c)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:59.818999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:00.034207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fac524cc46d4271b8f63e4a847af69c', 'request_id': 'req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:00.040073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fac524cc46d4271b8f63e4a847af69c', 'request_id': 'req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:00.047153 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fac524cc46d4271b8f63e4a847af69c', 'request_id': 'req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:00.163297 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] The token cache is being invalidate because project 6fac524cc46d4271b8f63e4a847af69c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:00.163871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:00.164675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a244df8a-df6c-4a23-bf6f-59ebff08e5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:00.166401 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1299/2590] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:24:59 2022] DELETE /identity/v3/projects/6fac524cc46d4271b8f63e4a847af69c => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:00.173336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84c0b-24cc-4d17-ba9c-9338f4057b80 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:00.173967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84c0b-24cc-4d17-ba9c-9338f4057b80 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:00.174255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84c0b-24cc-4d17-ba9c-9338f4057b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:00.235998 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-2fa84c0b-24cc-4d17-ba9c-9338f4057b80 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:00.237388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2fa84c0b-24cc-4d17-ba9c-9338f4057b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:00.346593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1292/2591] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:00.355244 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeb237-d3e6-4aa0-9e7e-9f2cf317567a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:00.355475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeb237-d3e6-4aa0-9e7e-9f2cf317567a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:00.355749 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeb237-d3e6-4aa0-9e7e-9f2cf317567a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:00.379235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-cdaeb237-d3e6-4aa0-9e7e-9f2cf317567a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:00.380706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-cdaeb237-d3e6-4aa0-9e7e-9f2cf317567a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:00.392485 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1300/2592] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:00.401383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa84c0b-24cc-4d17-ba9c-9338f4057b80 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:00.409441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa84c0b-24cc-4d17-ba9c-9338f4057b80 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:00.414563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4b54f-ed01-49d5-aab4-f9a19d5aaa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:00.415179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4b54f-ed01-49d5-aab4-f9a19d5aaa64 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:00.415459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4b54f-ed01-49d5-aab4-f9a19d5aaa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:00.415735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4b54f-ed01-49d5-aab4-f9a19d5aaa64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:00.416392 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4b54f-ed01-49d5-aab4-f9a19d5aaa64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:25:00.416699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4b54f-ed01-49d5-aab4-f9a19d5aaa64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:00.418683 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4b54f-ed01-49d5-aab4-f9a19d5aaa64 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:00.439811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1293/2593] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:25:00.444835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaeb237-d3e6-4aa0-9e7e-9f2cf317567a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:00.458374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaeb237-d3e6-4aa0-9e7e-9f2cf317567a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:00.465488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ba79e-9dbb-428e-92c7-43a6c6185973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:00.466688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ba79e-9dbb-428e-92c7-43a6c6185973 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:00.466688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ba79e-9dbb-428e-92c7-43a6c6185973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:00.466899 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ba79e-9dbb-428e-92c7-43a6c6185973 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:00.467437 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ba79e-9dbb-428e-92c7-43a6c6185973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:00.469429 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ba79e-9dbb-428e-92c7-43a6c6185973 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:00.587917 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4b54f-ed01-49d5-aab4-f9a19d5aaa64 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:00.588148 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1301/2594] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:00 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:00.603323 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4b54f-ed01-49d5-aab4-f9a19d5aaa64 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:00.681506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347a87a-9e04-4ce6-ac31-99c9eeff204d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:00.682301 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347a87a-9e04-4ce6-ac31-99c9eeff204d admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:00.682685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347a87a-9e04-4ce6-ac31-99c9eeff204d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:00.683049 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347a87a-9e04-4ce6-ac31-99c9eeff204d admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:00.683760 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347a87a-9e04-4ce6-ac31-99c9eeff204d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:00.686411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347a87a-9e04-4ce6-ac31-99c9eeff204d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:00.822418 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ba79e-9dbb-428e-92c7-43a6c6185973 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:00.824504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1294/2595] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:00 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:00.835036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ba79e-9dbb-428e-92c7-43a6c6185973 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:00.840558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86470a87-93d7-437f-bf59-d35a03d9b1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:00.841255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86470a87-93d7-437f-bf59-d35a03d9b1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:00.841606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86470a87-93d7-437f-bf59-d35a03d9b1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:00.841941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86470a87-93d7-437f-bf59-d35a03d9b1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:00.843024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86470a87-93d7-437f-bf59-d35a03d9b1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:00.845007 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86470a87-93d7-437f-bf59-d35a03d9b1d9 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:00.851363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:00.857205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1302/2596] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:00.861232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347a87a-9e04-4ce6-ac31-99c9eeff204d admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:00.878711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347a87a-9e04-4ce6-ac31-99c9eeff204d admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:00.888531 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db327d-e66a-44e6-bd84-2502a559db65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:00.888759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db327d-e66a-44e6-bd84-2502a559db65 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:00.889148 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db327d-e66a-44e6-bd84-2502a559db65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:00.890025 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db327d-e66a-44e6-bd84-2502a559db65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:00.892597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db327d-e66a-44e6-bd84-2502a559db65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:00.894719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db327d-e66a-44e6-bd84-2502a559db65 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:00.911224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1295/2597] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:00.916991 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86470a87-93d7-437f-bf59-d35a03d9b1d9 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:00.930916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86470a87-93d7-437f-bf59-d35a03d9b1d9 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:00.939375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779930b-2acf-4f9d-b564-f39d9c63e1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:00.943810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779930b-2acf-4f9d-b564-f39d9c63e1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:00.943810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779930b-2acf-4f9d-b564-f39d9c63e1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:00.943810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779930b-2acf-4f9d-b564-f39d9c63e1d4 admin admin] PATH_INFO: `/v3/projects/606ffd6c0fe24db692470cb7d37235a3/users/67195860373046f5869055f4d298218b/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:00.961827 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779930b-2acf-4f9d-b564-f39d9c63e1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=606ffd6c0fe24db692470cb7d37235a3, user_id=67195860373046f5869055f4d298218b, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:00.964081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779930b-2acf-4f9d-b564-f39d9c63e1d4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:01.204696 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1303/2598] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:00 2022] PUT /identity/v3/projects/606ffd6c0fe24db692470cb7d37235a3/users/67195860373046f5869055f4d298218b/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:01.218276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d384c1-77a8-4cc1-b8ef-cd76cd89658c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:01.219142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d384c1-77a8-4cc1-b8ef-cd76cd89658c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:01.219649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d384c1-77a8-4cc1-b8ef-cd76cd89658c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:01.252808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-92d384c1-77a8-4cc1-b8ef-cd76cd89658c None None] MFA Rules not processed for user `67195860373046f5869055f4d298218b`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:01.254222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-92d384c1-77a8-4cc1-b8ef-cd76cd89658c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:01.284099 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:01.347656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1296/2599] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:01.373449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779930b-2acf-4f9d-b564-f39d9c63e1d4 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:01.375565 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-d779930b-2acf-4f9d-b564-f39d9c63e1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:01.447669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779930b-2acf-4f9d-b564-f39d9c63e1d4 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:01.451335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df7a97-4a0f-49b8-80c3-ce09b88ad56b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:01.452596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df7a97-4a0f-49b8-80c3-ce09b88ad56b service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:01.452896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df7a97-4a0f-49b8-80c3-ce09b88ad56b service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:01.453171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df7a97-4a0f-49b8-80c3-ce09b88ad56b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:01.462115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df7a97-4a0f-49b8-80c3-ce09b88ad56b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:01.463642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df7a97-4a0f-49b8-80c3-ce09b88ad56b service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:01.473689 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1304/2600] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:01.503178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d384c1-77a8-4cc1-b8ef-cd76cd89658c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:01.571157 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d384c1-77a8-4cc1-b8ef-cd76cd89658c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:01.576302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:01.579841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:01.580117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:01.580389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] PATH_INFO: `/v3/users/67195860373046f5869055f4d298218b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:01.581645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67195860373046f5869055f4d298218b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:01.583822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:01.798308 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67195860373046f5869055f4d298218b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:01.799155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67195860373046f5869055f4d298218b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:01.807068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67195860373046f5869055f4d298218b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:01.820594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67195860373046f5869055f4d298218b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:01.980740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67195860373046f5869055f4d298218b'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:01.988447 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1297/2601] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:01 2022] DELETE /identity/v3/users/67195860373046f5869055f4d298218b => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:01.993212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c9c9f-4f32-49c1-8797-0f46bc50fcbf None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:01.993678 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c9c9f-4f32-49c1-8797-0f46bc50fcbf None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:01.993777 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c9c9f-4f32-49c1-8797-0f46bc50fcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:02.020493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-058c9c9f-4f32-49c1-8797-0f46bc50fcbf None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:02.030868 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-058c9c9f-4f32-49c1-8797-0f46bc50fcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:02.121697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1305/2602] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:02.136733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:02.149752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:02.219027 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:02.238730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fca88-5365-4f5e-a60a-b67aa2fd69f3 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:02.246472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5638cc-637b-44da-8a89-e60d50215531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:02.246472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5638cc-637b-44da-8a89-e60d50215531 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:02.246472 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5638cc-637b-44da-8a89-e60d50215531 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:02.246808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5638cc-637b-44da-8a89-e60d50215531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:02.264611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5638cc-637b-44da-8a89-e60d50215531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:02.264611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5638cc-637b-44da-8a89-e60d50215531 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:02.278015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1298/2603] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:03.005158 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c9c9f-4f32-49c1-8797-0f46bc50fcbf None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:03.006633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-058c9c9f-4f32-49c1-8797-0f46bc50fcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:03.025995 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c9c9f-4f32-49c1-8797-0f46bc50fcbf None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:03.036941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:03.036941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:03.036941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:03.036941 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] PATH_INFO: `/v3/projects/606ffd6c0fe24db692470cb7d37235a3` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:03.038505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=606ffd6c0fe24db692470cb7d37235a3)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:03.039420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:03.135173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5638cc-637b-44da-8a89-e60d50215531 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:03.136473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-db5638cc-637b-44da-8a89-e60d50215531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:03.151506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5638cc-637b-44da-8a89-e60d50215531 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:03.158489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:03.159135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:03.159404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:03.159867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] PATH_INFO: `/v3/users/8d72e2cc5b804b0bb556599a26d923ac` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:03.161045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d72e2cc5b804b0bb556599a26d923ac)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:03.163564 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:03.285230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '606ffd6c0fe24db692470cb7d37235a3', 'request_id': 'req-deb73afd-bf3f-402b-9755-c652a6db48cc'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:03.290937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '606ffd6c0fe24db692470cb7d37235a3', 'request_id': 'req-deb73afd-bf3f-402b-9755-c652a6db48cc'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:03.298385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '606ffd6c0fe24db692470cb7d37235a3', 'request_id': 'req-deb73afd-bf3f-402b-9755-c652a6db48cc'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:03.460894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] The token cache is being invalidate because project 606ffd6c0fe24db692470cb7d37235a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:03.461227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:03.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-deb73afd-bf3f-402b-9755-c652a6db48cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:03.468209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1306/2604] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:02 2022] DELETE /identity/v3/projects/606ffd6c0fe24db692470cb7d37235a3 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:03.470184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae18d0-1ed2-4664-8abe-58b57e7d01d5 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:03.470474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae18d0-1ed2-4664-8abe-58b57e7d01d5 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:03.470910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae18d0-1ed2-4664-8abe-58b57e7d01d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:03.474043 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d72e2cc5b804b0bb556599a26d923ac'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:03.474985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d72e2cc5b804b0bb556599a26d923ac'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:03.478760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:03.484460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d72e2cc5b804b0bb556599a26d923ac'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:03.492024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-85ae18d0-1ed2-4664-8abe-58b57e7d01d5 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:03.493274 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-85ae18d0-1ed2-4664-8abe-58b57e7d01d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:03.504825 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d72e2cc5b804b0bb556599a26d923ac'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:03.565711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1307/2605] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:03.573054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f773-e2e4-43d1-b8a8-bd8e4f3f0b67 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:03.573339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f773-e2e4-43d1-b8a8-bd8e4f3f0b67 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:03.573613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f773-e2e4-43d1-b8a8-bd8e4f3f0b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:03.593529 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-4213f773-e2e4-43d1-b8a8-bd8e4f3f0b67 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:03.594714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-4213f773-e2e4-43d1-b8a8-bd8e4f3f0b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:03.602702 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1308/2606] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:03.606745 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213f773-e2e4-43d1-b8a8-bd8e4f3f0b67 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:03.609437 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d72e2cc5b804b0bb556599a26d923ac'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:03.616670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1299/2607] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:03 2022] DELETE /identity/v3/users/8d72e2cc5b804b0bb556599a26d923ac => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:03.624972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213f773-e2e4-43d1-b8a8-bd8e4f3f0b67 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:03.677826 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b980dd-dbb4-4f3f-a074-b2cb950d5b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:03.678432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b980dd-dbb4-4f3f-a074-b2cb950d5b69 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:03.678710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b980dd-dbb4-4f3f-a074-b2cb950d5b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:03.678980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b980dd-dbb4-4f3f-a074-b2cb950d5b69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:03.679647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b980dd-dbb4-4f3f-a074-b2cb950d5b69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:25:03.679946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b980dd-dbb4-4f3f-a074-b2cb950d5b69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:03.681856 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b980dd-dbb4-4f3f-a074-b2cb950d5b69 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:03.694361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1309/2608] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:25:03.698403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:03.699819 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:03.713542 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b68bdf-a98e-45ce-a05f-f968dd895bde admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:03.719389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cdc2d-781d-4fe9-a13c-d7f2ab31546e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:03.720018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cdc2d-781d-4fe9-a13c-d7f2ab31546e admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:03.720300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cdc2d-781d-4fe9-a13c-d7f2ab31546e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:03.720627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cdc2d-781d-4fe9-a13c-d7f2ab31546e admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:03.721321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cdc2d-781d-4fe9-a13c-d7f2ab31546e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:03.723649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cdc2d-781d-4fe9-a13c-d7f2ab31546e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:03.868053 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1300/2609] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:03 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:03.872129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b980dd-dbb4-4f3f-a074-b2cb950d5b69 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:03.882944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b980dd-dbb4-4f3f-a074-b2cb950d5b69 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:03.908915 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cdc2d-781d-4fe9-a13c-d7f2ab31546e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:03.911743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a06cdc2d-781d-4fe9-a13c-d7f2ab31546e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:03.970606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e9717-0175-479d-9342-248ca01331ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:03.971218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e9717-0175-479d-9342-248ca01331ad admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:03.971493 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e9717-0175-479d-9342-248ca01331ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:03.971768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e9717-0175-479d-9342-248ca01331ad admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:03.972381 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e9717-0175-479d-9342-248ca01331ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:03.974325 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e9717-0175-479d-9342-248ca01331ad admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:04.003547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cdc2d-781d-4fe9-a13c-d7f2ab31546e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:04.008321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:04.014360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.014360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.014360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] PATH_INFO: `/v3/projects/dad90b6c7dd947dda6faa1fc1feb6d7b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.014360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad90b6c7dd947dda6faa1fc1feb6d7b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:04.014360 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:04.108757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1310/2610] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:03 2022] POST /identity/v3/users => generated 287 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:04.113214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e9717-0175-479d-9342-248ca01331ad admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:04.123470 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e9717-0175-479d-9342-248ca01331ad admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:04.128195 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f48dd-40f3-4596-931e-0fe9354f8613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:04.128774 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f48dd-40f3-4596-931e-0fe9354f8613 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.129082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f48dd-40f3-4596-931e-0fe9354f8613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.129362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f48dd-40f3-4596-931e-0fe9354f8613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.130020 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f48dd-40f3-4596-931e-0fe9354f8613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:04.131675 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f48dd-40f3-4596-931e-0fe9354f8613 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:04.142963 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1311/2611] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:25:04.147240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f48dd-40f3-4596-931e-0fe9354f8613 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:04.152601 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:04.158586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f48dd-40f3-4596-931e-0fe9354f8613 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:04.163367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:04.165714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad90b6c7dd947dda6faa1fc1feb6d7b', 'request_id': 'req-60aeaf72-9883-4366-829d-283e8b42fc25'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:04.171595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad90b6c7dd947dda6faa1fc1feb6d7b', 'request_id': 'req-60aeaf72-9883-4366-829d-283e8b42fc25'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:04.216367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8aca7c-8840-402a-a5e1-c881326ff67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:04.217041 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aca7c-8840-402a-a5e1-c881326ff67a admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.217988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aca7c-8840-402a-a5e1-c881326ff67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.218196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aca7c-8840-402a-a5e1-c881326ff67a admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.219008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8aca7c-8840-402a-a5e1-c881326ff67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:04.220582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8aca7c-8840-402a-a5e1-c881326ff67a admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:04.231128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1312/2612] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:25:04.236360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8aca7c-8840-402a-a5e1-c881326ff67a admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:04.245959 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8aca7c-8840-402a-a5e1-c881326ff67a admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:04.251567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b2837-5039-48dc-8606-845a00489c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:04.252230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b2837-5039-48dc-8606-845a00489c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.258337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b2837-5039-48dc-8606-845a00489c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.258337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b2837-5039-48dc-8606-845a00489c36 admin admin] PATH_INFO: `/v3/projects/ba4ba831a31648e4a3fbba24ad744780/users/8b2a8ab882af469ea1634c579c82cad9/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.262271 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad90b6c7dd947dda6faa1fc1feb6d7b', 'request_id': 'req-60aeaf72-9883-4366-829d-283e8b42fc25'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:04.271133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b2837-5039-48dc-8606-845a00489c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba4ba831a31648e4a3fbba24ad744780, user_id=8b2a8ab882af469ea1634c579c82cad9, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:04.273824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b2837-5039-48dc-8606-845a00489c36 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:04.275649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] The token cache is being invalidate because project dad90b6c7dd947dda6faa1fc1feb6d7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:04.276042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:04.276622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-60aeaf72-9883-4366-829d-283e8b42fc25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:04.278494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1301/2613] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:03 2022] DELETE /identity/v3/projects/dad90b6c7dd947dda6faa1fc1feb6d7b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:04.284035 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aa790-6900-4eca-98ce-ee3ac6fa3ba4 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.284327 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aa790-6900-4eca-98ce-ee3ac6fa3ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.284605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aa790-6900-4eca-98ce-ee3ac6fa3ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.304535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ea5aa790-6900-4eca-98ce-ee3ac6fa3ba4 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:04.305807 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ea5aa790-6900-4eca-98ce-ee3ac6fa3ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:04.314121 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1302/2614] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:04.322289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee7dd4-f3f5-48a3-941c-2fd73352bc35 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.322579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee7dd4-f3f5-48a3-941c-2fd73352bc35 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.322859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee7dd4-f3f5-48a3-941c-2fd73352bc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.343124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-6cee7dd4-f3f5-48a3-941c-2fd73352bc35 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:04.344348 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6cee7dd4-f3f5-48a3-941c-2fd73352bc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:04.352779 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1303/2615] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:04.356933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee7dd4-f3f5-48a3-941c-2fd73352bc35 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:04.367275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee7dd4-f3f5-48a3-941c-2fd73352bc35 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:04.372343 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94593f45-a547-4216-9852-04cc14028208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:04.372992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94593f45-a547-4216-9852-04cc14028208 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.373279 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94593f45-a547-4216-9852-04cc14028208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.373590 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94593f45-a547-4216-9852-04cc14028208 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.374562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94593f45-a547-4216-9852-04cc14028208 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:25:04.374867 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94593f45-a547-4216-9852-04cc14028208 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:04.376482 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94593f45-a547-4216-9852-04cc14028208 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:04.388193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1304/2616] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:25:04.393065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94593f45-a547-4216-9852-04cc14028208 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:04.403357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94593f45-a547-4216-9852-04cc14028208 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:04.411652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00568a-c269-424c-a82d-a391d629ced8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:04.412245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00568a-c269-424c-a82d-a391d629ced8 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.412525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00568a-c269-424c-a82d-a391d629ced8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.412798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00568a-c269-424c-a82d-a391d629ced8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.413458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00568a-c269-424c-a82d-a391d629ced8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:04.415349 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00568a-c269-424c-a82d-a391d629ced8 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:04.432499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1313/2617] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:04 2022] PUT /identity/v3/projects/ba4ba831a31648e4a3fbba24ad744780/users/8b2a8ab882af469ea1634c579c82cad9/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:04.443549 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5ec93-7e24-4490-8315-8974d576a595 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.443838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5ec93-7e24-4490-8315-8974d576a595 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.444119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5ec93-7e24-4490-8315-8974d576a595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.472481 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-89b5ec93-7e24-4490-8315-8974d576a595 None None] MFA Rules not processed for user `8b2a8ab882af469ea1634c579c82cad9`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:04.473662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-89b5ec93-7e24-4490-8315-8974d576a595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:04.538050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1314/2618] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:04.558187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5ec93-7e24-4490-8315-8974d576a595 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:04.558187 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-89b5ec93-7e24-4490-8315-8974d576a595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:04.566497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1305/2619] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:04 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:25:04.569571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00568a-c269-424c-a82d-a391d629ced8 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:04.628685 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5ec93-7e24-4490-8315-8974d576a595 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:04.653449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00568a-c269-424c-a82d-a391d629ced8 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:04.659746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:04.660066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bce09-2cf3-41f5-867d-00510afd4cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:04.660778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bce09-2cf3-41f5-867d-00510afd4cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.661161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bce09-2cf3-41f5-867d-00510afd4cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.661528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bce09-2cf3-41f5-867d-00510afd4cc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.662345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bce09-2cf3-41f5-867d-00510afd4cc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:04.664657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bce09-2cf3-41f5-867d-00510afd4cc0 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:04.691655 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443fe1a-2845-426f-98e6-955a295b5036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:04.698110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443fe1a-2845-426f-98e6-955a295b5036 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.698110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443fe1a-2845-426f-98e6-955a295b5036 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.698110 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443fe1a-2845-426f-98e6-955a295b5036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.770236 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1306/2620] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:04 2022] POST /identity/v3/users => generated 284 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:25:04.770891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443fe1a-2845-426f-98e6-955a295b5036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:04.771784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bce09-2cf3-41f5-867d-00510afd4cc0 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:04.772740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443fe1a-2845-426f-98e6-955a295b5036 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:04.786945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bce09-2cf3-41f5-867d-00510afd4cc0 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:04.789910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1315/2621] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:25:04.792443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94844e8e-cd3f-44cb-bc24-972bb3633122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:04.793188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94844e8e-cd3f-44cb-bc24-972bb3633122 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.793670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94844e8e-cd3f-44cb-bc24-972bb3633122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.794171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94844e8e-cd3f-44cb-bc24-972bb3633122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.795580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94844e8e-cd3f-44cb-bc24-972bb3633122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:04.797690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94844e8e-cd3f-44cb-bc24-972bb3633122 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:04.809514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1307/2622] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:04.814360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443fe1a-2845-426f-98e6-955a295b5036 service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:04.825293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443fe1a-2845-426f-98e6-955a295b5036 service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:04.830789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b1638-3b3d-4f63-a84c-a64e2389ef2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:04.831430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b1638-3b3d-4f63-a84c-a64e2389ef2b admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.837003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b1638-3b3d-4f63-a84c-a64e2389ef2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.837003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b1638-3b3d-4f63-a84c-a64e2389ef2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.837003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b1638-3b3d-4f63-a84c-a64e2389ef2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:04.837003 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b1638-3b3d-4f63-a84c-a64e2389ef2b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:04.859761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1316/2623] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:25:04.863845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94844e8e-cd3f-44cb-bc24-972bb3633122 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:04.878224 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94844e8e-cd3f-44cb-bc24-972bb3633122 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:04.884467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6099d-d7af-4c1a-bb8d-ab7cb459faa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:04.885222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6099d-d7af-4c1a-bb8d-ab7cb459faa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:04.885526 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6099d-d7af-4c1a-bb8d-ab7cb459faa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:04.885812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6099d-d7af-4c1a-bb8d-ab7cb459faa6 admin admin] PATH_INFO: `/v3/projects/4168407f69c14855ac56a9f0f5ef2b9e/users/2605871ced3a4c5aa40ea3a599e7f45e/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:04.901981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6099d-d7af-4c1a-bb8d-ab7cb459faa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4168407f69c14855ac56a9f0f5ef2b9e, user_id=2605871ced3a4c5aa40ea3a599e7f45e, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:04.904764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6099d-d7af-4c1a-bb8d-ab7cb459faa6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:05.027298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1308/2624] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:04 2022] PUT /identity/v3/projects/4168407f69c14855ac56a9f0f5ef2b9e/users/2605871ced3a4c5aa40ea3a599e7f45e/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:05.039757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2d1be-f280-4e6e-82af-82e86cd8c123 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:05.040329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2d1be-f280-4e6e-82af-82e86cd8c123 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:05.040329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2d1be-f280-4e6e-82af-82e86cd8c123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:05.070464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-dbd2d1be-f280-4e6e-82af-82e86cd8c123 None None] MFA Rules not processed for user `2605871ced3a4c5aa40ea3a599e7f45e`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:05.071860 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-dbd2d1be-f280-4e6e-82af-82e86cd8c123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:05.142669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1317/2625] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:05.153928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6099d-d7af-4c1a-bb8d-ab7cb459faa6 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:05.192187 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:05.218436 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6099d-d7af-4c1a-bb8d-ab7cb459faa6 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:05.222047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d650b0-8c8f-4857-903e-146c17fd6961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:05.222644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d650b0-8c8f-4857-903e-146c17fd6961 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:05.222923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d650b0-8c8f-4857-903e-146c17fd6961 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:05.223193 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d650b0-8c8f-4857-903e-146c17fd6961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:05.231891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d650b0-8c8f-4857-903e-146c17fd6961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:05.233663 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d650b0-8c8f-4857-903e-146c17fd6961 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:05.242910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1309/2626] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:15.138302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=73 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:15.735669 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=14 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:17.482382 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2d1be-f280-4e6e-82af-82e86cd8c123 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:17.548467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2d1be-f280-4e6e-82af-82e86cd8c123 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:17.552818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:17.553492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:17.553768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:17.554038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] PATH_INFO: `/v3/users/2605871ced3a4c5aa40ea3a599e7f45e` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:17.555136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=2605871ced3a4c5aa40ea3a599e7f45e)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:17.557056 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:17.789180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2605871ced3a4c5aa40ea3a599e7f45e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:17.789943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2605871ced3a4c5aa40ea3a599e7f45e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:17.798009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2605871ced3a4c5aa40ea3a599e7f45e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:17.798863 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2605871ced3a4c5aa40ea3a599e7f45e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:17.808127 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d650b0-8c8f-4857-903e-146c17fd6961 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:17.809686 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b5d650b0-8c8f-4857-903e-146c17fd6961 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:17.825011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d650b0-8c8f-4857-903e-146c17fd6961 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:17.829966 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e61cf-d123-4ffe-bf96-d81cfb29bab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:17.830637 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e61cf-d123-4ffe-bf96-d81cfb29bab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:17.830900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e61cf-d123-4ffe-bf96-d81cfb29bab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:17.831174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e61cf-d123-4ffe-bf96-d81cfb29bab3 admin admin] PATH_INFO: `/v3/users/8b2a8ab882af469ea1634c579c82cad9` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:17.832311 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e61cf-d123-4ffe-bf96-d81cfb29bab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2a8ab882af469ea1634c579c82cad9)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:17.834321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e61cf-d123-4ffe-bf96-d81cfb29bab3 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:17.951862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2605871ced3a4c5aa40ea3a599e7f45e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:17.971557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1318/2627] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:17 2022] DELETE /identity/v3/users/2605871ced3a4c5aa40ea3a599e7f45e => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:17.972408 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:17.973958 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:18.046245 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27111bf-4e7f-40cc-ac9f-cbaff969d6ed admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:18.051322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:18.051887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:18.052169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:18.052440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] PATH_INFO: `/v3/users/fde2b88311be4b58b1cf9efc1806e2c1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:18.053649 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde2b88311be4b58b1cf9efc1806e2c1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:18.055605 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:18.078614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:18.215546 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c41e61cf-d123-4ffe-bf96-d81cfb29bab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2a8ab882af469ea1634c579c82cad9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:18.216338 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c41e61cf-d123-4ffe-bf96-d81cfb29bab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2a8ab882af469ea1634c579c82cad9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:18.224849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c41e61cf-d123-4ffe-bf96-d81cfb29bab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2a8ab882af469ea1634c579c82cad9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:18.237544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c41e61cf-d123-4ffe-bf96-d81cfb29bab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2a8ab882af469ea1634c579c82cad9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:18.300318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c41e61cf-d123-4ffe-bf96-d81cfb29bab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2a8ab882af469ea1634c579c82cad9'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:18.309895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1310/2628] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:17 2022] DELETE /identity/v3/users/8b2a8ab882af469ea1634c579c82cad9 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:18.309895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960446a-f93d-40b5-89e4-09e2be247ba8 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:18.309895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960446a-f93d-40b5-89e4-09e2be247ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:18.309895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960446a-f93d-40b5-89e4-09e2be247ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:18.331574 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-d960446a-f93d-40b5-89e4-09e2be247ba8 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:18.333371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d960446a-f93d-40b5-89e4-09e2be247ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:18.333694 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde2b88311be4b58b1cf9efc1806e2c1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:18.334370 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde2b88311be4b58b1cf9efc1806e2c1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:18.342240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde2b88311be4b58b1cf9efc1806e2c1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:18.342942 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde2b88311be4b58b1cf9efc1806e2c1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:18.406579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1311/2629] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:18.408103 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4465714-394d-43cf-9c0d-48dde4d7c12c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:18.408389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4465714-394d-43cf-9c0d-48dde4d7c12c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:18.408732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4465714-394d-43cf-9c0d-48dde4d7c12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:18.431342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-f4465714-394d-43cf-9c0d-48dde4d7c12c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:18.432627 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f4465714-394d-43cf-9c0d-48dde4d7c12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:18.439461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde2b88311be4b58b1cf9efc1806e2c1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:18.441770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1312/2630] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:18.442619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4465714-394d-43cf-9c0d-48dde4d7c12c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:18.443906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f4465714-394d-43cf-9c0d-48dde4d7c12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:18.467889 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1319/2631] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:17 2022] DELETE /identity/v3/users/fde2b88311be4b58b1cf9efc1806e2c1 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:18.468658 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:18.470283 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:18.528799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:18.533418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4465714-394d-43cf-9c0d-48dde4d7c12c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:18.537157 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddd266-98eb-4eed-90c0-09f9009bdfa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:18.537688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009b967-b6ac-443f-9d08-bf54f54ac164 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:18.538233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddd266-98eb-4eed-90c0-09f9009bdfa5 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:18.538233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddd266-98eb-4eed-90c0-09f9009bdfa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:18.538614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddd266-98eb-4eed-90c0-09f9009bdfa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:18.541537 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c503af9-0017-4879-8a4f-40624fff47af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:18.542147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c503af9-0017-4879-8a4f-40624fff47af service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:18.542427 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c503af9-0017-4879-8a4f-40624fff47af service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:18.542704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c503af9-0017-4879-8a4f-40624fff47af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:18.602117 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddd266-98eb-4eed-90c0-09f9009bdfa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:18.603571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddd266-98eb-4eed-90c0-09f9009bdfa5 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:18.610840 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c503af9-0017-4879-8a4f-40624fff47af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:18.612293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c503af9-0017-4879-8a4f-40624fff47af service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:18.615017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1313/2632] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:18.616289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c214f-2206-4f9b-8c64-66d27e5b076e None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:18.616572 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c214f-2206-4f9b-8c64-66d27e5b076e None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:18.616891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c214f-2206-4f9b-8c64-66d27e5b076e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:18.624292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1320/2633] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:18.638216 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-c33c214f-2206-4f9b-8c64-66d27e5b076e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:18.639458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c33c214f-2206-4f9b-8c64-66d27e5b076e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:18.648591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1314/2634] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:18.662925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c503af9-0017-4879-8a4f-40624fff47af service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:18.674200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c503af9-0017-4879-8a4f-40624fff47af service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:18.678335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44443312-1f8d-4e52-8288-e4fc499b830d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:18.678952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44443312-1f8d-4e52-8288-e4fc499b830d service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:18.679238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44443312-1f8d-4e52-8288-e4fc499b830d service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:18.679647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44443312-1f8d-4e52-8288-e4fc499b830d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:18.690146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44443312-1f8d-4e52-8288-e4fc499b830d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:18.691871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44443312-1f8d-4e52-8288-e4fc499b830d service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:18.703335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1321/2635] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:19.179709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c214f-2206-4f9b-8c64-66d27e5b076e None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:19.181284 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c33c214f-2206-4f9b-8c64-66d27e5b076e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:19.194283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c214f-2206-4f9b-8c64-66d27e5b076e None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:19.199206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:19.199842 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:19.200248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:19.200532 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] PATH_INFO: `/v3/projects/4168407f69c14855ac56a9f0f5ef2b9e` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:19.201783 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4168407f69c14855ac56a9f0f5ef2b9e)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:19.204230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:19.247835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44443312-1f8d-4e52-8288-e4fc499b830d service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:19.249345 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-44443312-1f8d-4e52-8288-e4fc499b830d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:19.262805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44443312-1f8d-4e52-8288-e4fc499b830d service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:19.269069 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:19.269717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:19.270001 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:19.270280 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] PATH_INFO: `/v3/projects/ba4ba831a31648e4a3fbba24ad744780` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:19.271492 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba4ba831a31648e4a3fbba24ad744780)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:19.277644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:19.299609 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:19.381757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4168407f69c14855ac56a9f0f5ef2b9e', 'request_id': 'req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.387699 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4168407f69c14855ac56a9f0f5ef2b9e', 'request_id': 'req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.416076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba4ba831a31648e4a3fbba24ad744780', 'request_id': 'req-92447fd6-f727-487a-b053-a8b27431d239'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.422353 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba4ba831a31648e4a3fbba24ad744780', 'request_id': 'req-92447fd6-f727-487a-b053-a8b27431d239'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.429320 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba4ba831a31648e4a3fbba24ad744780', 'request_id': 'req-92447fd6-f727-487a-b053-a8b27431d239'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.461643 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4168407f69c14855ac56a9f0f5ef2b9e', 'request_id': 'req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.474304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] The token cache is being invalidate because project 4168407f69c14855ac56a9f0f5ef2b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:19.474505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.475037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.476537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1315/2636] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:19 2022] DELETE /identity/v3/projects/4168407f69c14855ac56a9f0f5ef2b9e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:19.477353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:19.478607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:19.543511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1c4a6-e4bd-48a3-8f17-dddc4277a40a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:19.559841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:19.585857 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] The token cache is being invalidate because project ba4ba831a31648e4a3fbba24ad744780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:19.586450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.586784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-92447fd6-f727-487a-b053-a8b27431d239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.588340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1322/2637] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:19 2022] DELETE /identity/v3/projects/ba4ba831a31648e4a3fbba24ad744780 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:19.590669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ba46d-0880-4848-a714-a1b18de72e61 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:19.590669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ba46d-0880-4848-a714-a1b18de72e61 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:19.590669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ba46d-0880-4848-a714-a1b18de72e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:19.602925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:19.603569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:19.603850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:19.604247 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] PATH_INFO: `/v3/projects/9cf504ae1ab34f86923979670ac1c013` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:19.605416 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cf504ae1ab34f86923979670ac1c013)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:19.607311 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:19.610295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-884ba46d-0880-4848-a714-a1b18de72e61 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:19.617027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-884ba46d-0880-4848-a714-a1b18de72e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:19.619750 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1323/2638] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:19.621133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff26b2-8663-4e1b-813e-74f717f32d4a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:19.621347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff26b2-8663-4e1b-813e-74f717f32d4a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:19.621584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff26b2-8663-4e1b-813e-74f717f32d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:19.643221 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-09ff26b2-8663-4e1b-813e-74f717f32d4a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:19.644419 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-09ff26b2-8663-4e1b-813e-74f717f32d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:19.652200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1324/2639] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:19.653385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf110f2e-606c-43a0-a60e-de37c5578e30 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:19.659149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf110f2e-606c-43a0-a60e-de37c5578e30 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:19.659149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf110f2e-606c-43a0-a60e-de37c5578e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:19.672815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-cf110f2e-606c-43a0-a60e-de37c5578e30 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:19.673955 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-cf110f2e-606c-43a0-a60e-de37c5578e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:19.681729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1325/2640] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:19.682862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b3f71-4e10-4f0e-8bf3-b3041bf055e3 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:19.683141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b3f71-4e10-4f0e-8bf3-b3041bf055e3 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:19.683406 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b3f71-4e10-4f0e-8bf3-b3041bf055e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:19.703045 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-df7b3f71-4e10-4f0e-8bf3-b3041bf055e3 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:19.705203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-df7b3f71-4e10-4f0e-8bf3-b3041bf055e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:19.713135 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1326/2641] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:19.713960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b3f71-4e10-4f0e-8bf3-b3041bf055e3 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:19.723728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b3f71-4e10-4f0e-8bf3-b3041bf055e3 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:19.728557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aba616-6b3d-4ca9-b970-ceec265e71b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:19.739308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aba616-6b3d-4ca9-b970-ceec265e71b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:19.739308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aba616-6b3d-4ca9-b970-ceec265e71b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:19.739308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aba616-6b3d-4ca9-b970-ceec265e71b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:19.739308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aba616-6b3d-4ca9-b970-ceec265e71b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:25:19.739308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aba616-6b3d-4ca9-b970-ceec265e71b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:19.739308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aba616-6b3d-4ca9-b970-ceec265e71b8 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:19.741572 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cf504ae1ab34f86923979670ac1c013', 'request_id': 'req-94145ece-a11e-4b3a-82c3-67f9dd60f002'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.745337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1327/2642] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:25:19.745337 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aba616-6b3d-4ca9-b970-ceec265e71b8 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:19.749142 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cf504ae1ab34f86923979670ac1c013', 'request_id': 'req-94145ece-a11e-4b3a-82c3-67f9dd60f002'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.756926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aba616-6b3d-4ca9-b970-ceec265e71b8 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:19.812499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cf68a-5e17-41cd-b957-63d01daac91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:19.813786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf68a-5e17-41cd-b957-63d01daac91b admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:19.814188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf68a-5e17-41cd-b957-63d01daac91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:19.814463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf68a-5e17-41cd-b957-63d01daac91b admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:19.815142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cf68a-5e17-41cd-b957-63d01daac91b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:25:19.815446 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cf68a-5e17-41cd-b957-63d01daac91b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:19.817054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cf68a-5e17-41cd-b957-63d01daac91b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:19.829296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1328/2643] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:25:19.830441 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cf68a-5e17-41cd-b957-63d01daac91b admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:19.840188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cf68a-5e17-41cd-b957-63d01daac91b admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:19.845349 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4287b1-9d64-497a-9ce5-22e3a7ddd2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:19.845956 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4287b1-9d64-497a-9ce5-22e3a7ddd2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:19.846234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4287b1-9d64-497a-9ce5-22e3a7ddd2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:19.847209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4287b1-9d64-497a-9ce5-22e3a7ddd2e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:19.847839 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4287b1-9d64-497a-9ce5-22e3a7ddd2e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:19.849768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4287b1-9d64-497a-9ce5-22e3a7ddd2e5 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:19.861059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cf504ae1ab34f86923979670ac1c013', 'request_id': 'req-94145ece-a11e-4b3a-82c3-67f9dd60f002'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.874166 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] The token cache is being invalidate because project 9cf504ae1ab34f86923979670ac1c013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:19.874489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.875173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:19.880897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1316/2644] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:19 2022] DELETE /identity/v3/projects/9cf504ae1ab34f86923979670ac1c013 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:19.880897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:19.880897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:19.893539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94145ece-a11e-4b3a-82c3-67f9dd60f002 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:19.898524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e35ac-58dc-4e56-998a-7fd243fad4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:19.899140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e35ac-58dc-4e56-998a-7fd243fad4df admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:19.899421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e35ac-58dc-4e56-998a-7fd243fad4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:19.899697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e35ac-58dc-4e56-998a-7fd243fad4df admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:19.900352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e35ac-58dc-4e56-998a-7fd243fad4df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:19.902517 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e35ac-58dc-4e56-998a-7fd243fad4df admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:19.960566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1329/2645] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:19 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:19.965269 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4287b1-9d64-497a-9ce5-22e3a7ddd2e5 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:19.967012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-bb4287b1-9d64-497a-9ce5-22e3a7ddd2e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:20.019489 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:20.040467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4287b1-9d64-497a-9ce5-22e3a7ddd2e5 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.045480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df9498-73f2-4e65-9a95-071552cd1dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.046151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df9498-73f2-4e65-9a95-071552cd1dfd admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.046431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df9498-73f2-4e65-9a95-071552cd1dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.046726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df9498-73f2-4e65-9a95-071552cd1dfd admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.048380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df9498-73f2-4e65-9a95-071552cd1dfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.050454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df9498-73f2-4e65-9a95-071552cd1dfd admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.058089 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1317/2646] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:19 2022] POST /identity/v3/projects => generated 397 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:20.062937 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e35ac-58dc-4e56-998a-7fd243fad4df admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.133202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e35ac-58dc-4e56-998a-7fd243fad4df admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.138034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a7cd4-0a73-419b-943c-b5671e69595f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.138661 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a7cd4-0a73-419b-943c-b5671e69595f admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.138959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a7cd4-0a73-419b-943c-b5671e69595f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.139220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a7cd4-0a73-419b-943c-b5671e69595f admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.139903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a7cd4-0a73-419b-943c-b5671e69595f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.141930 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a7cd4-0a73-419b-943c-b5671e69595f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.221287 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1330/2647] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:19 2022] POST /identity/v3/users => generated 283 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:20.225242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df9498-73f2-4e65-9a95-071552cd1dfd admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.235622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df9498-73f2-4e65-9a95-071552cd1dfd admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.241426 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768f6c5-9dbd-41da-a27d-34b42ca26346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.242032 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768f6c5-9dbd-41da-a27d-34b42ca26346 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.242314 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768f6c5-9dbd-41da-a27d-34b42ca26346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.242587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768f6c5-9dbd-41da-a27d-34b42ca26346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.243336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768f6c5-9dbd-41da-a27d-34b42ca26346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.244948 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768f6c5-9dbd-41da-a27d-34b42ca26346 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.256250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1331/2648] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:25:20.260553 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768f6c5-9dbd-41da-a27d-34b42ca26346 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.269209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1318/2649] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:20 2022] POST /identity/v3/users => generated 299 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:20.270341 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a7cd4-0a73-419b-943c-b5671e69595f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.272152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768f6c5-9dbd-41da-a27d-34b42ca26346 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.276895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6856f-c71c-4125-968c-e5e63fa4a9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.277519 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6856f-c71c-4125-968c-e5e63fa4a9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.277806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6856f-c71c-4125-968c-e5e63fa4a9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.278080 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6856f-c71c-4125-968c-e5e63fa4a9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.278724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6856f-c71c-4125-968c-e5e63fa4a9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.280396 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6856f-c71c-4125-968c-e5e63fa4a9f9 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.280568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a7cd4-0a73-419b-943c-b5671e69595f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.286250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea53eb5-de0b-420a-8741-2189c65803fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.287054 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea53eb5-de0b-420a-8741-2189c65803fd admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.287230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea53eb5-de0b-420a-8741-2189c65803fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.287506 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea53eb5-de0b-420a-8741-2189c65803fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.288151 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea53eb5-de0b-420a-8741-2189c65803fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.289796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea53eb5-de0b-420a-8741-2189c65803fd admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.300467 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1332/2650] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:25:20.304594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1319/2651] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:25:20.304751 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6856f-c71c-4125-968c-e5e63fa4a9f9 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.305537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea53eb5-de0b-420a-8741-2189c65803fd admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.316412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea53eb5-de0b-420a-8741-2189c65803fd admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.316902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6856f-c71c-4125-968c-e5e63fa4a9f9 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.322220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc727b-071e-4ab2-bb73-33c26aa1fabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.322820 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc727b-071e-4ab2-bb73-33c26aa1fabd admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.323113 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc727b-071e-4ab2-bb73-33c26aa1fabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.323385 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc727b-071e-4ab2-bb73-33c26aa1fabd admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.323887 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79faedc0-b1f3-4cfd-8824-7ad047da9247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.324014 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc727b-071e-4ab2-bb73-33c26aa1fabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.324543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faedc0-b1f3-4cfd-8824-7ad047da9247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.324822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faedc0-b1f3-4cfd-8824-7ad047da9247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.325129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faedc0-b1f3-4cfd-8824-7ad047da9247 admin admin] PATH_INFO: `/v3/projects/4e389a29436c43358c61d8cc7535d1af/users/638fe693e1a440588bcd3a546e93ec77/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.325659 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc727b-071e-4ab2-bb73-33c26aa1fabd admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.330629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:20.338985 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1320/2652] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:20.340789 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79faedc0-b1f3-4cfd-8824-7ad047da9247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e389a29436c43358c61d8cc7535d1af, user_id=638fe693e1a440588bcd3a546e93ec77, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.343114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc727b-071e-4ab2-bb73-33c26aa1fabd admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.343352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79faedc0-b1f3-4cfd-8824-7ad047da9247 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.354688 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc727b-071e-4ab2-bb73-33c26aa1fabd admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.360549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f560c3-5457-4bf5-befd-547d5d4159fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.361530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f560c3-5457-4bf5-befd-547d5d4159fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.361816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f560c3-5457-4bf5-befd-547d5d4159fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.362082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f560c3-5457-4bf5-befd-547d5d4159fd admin admin] PATH_INFO: `/v3/projects/8393d07980604e158f8ef4d6de702734/users/172c871291504ff19d70d020ec6c8cce/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.378997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f560c3-5457-4bf5-befd-547d5d4159fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8393d07980604e158f8ef4d6de702734, user_id=172c871291504ff19d70d020ec6c8cce, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.381745 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f560c3-5457-4bf5-befd-547d5d4159fd admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.427768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1333/2653] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:20 2022] PUT /identity/v3/projects/4e389a29436c43358c61d8cc7535d1af/users/638fe693e1a440588bcd3a546e93ec77/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:20.438251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd5c14-ce22-4d70-9644-c725c67cd74e None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.438531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd5c14-ce22-4d70-9644-c725c67cd74e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.438802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd5c14-ce22-4d70-9644-c725c67cd74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.467809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e5dd5c14-ce22-4d70-9644-c725c67cd74e None None] MFA Rules not processed for user `638fe693e1a440588bcd3a546e93ec77`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:20.469040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e5dd5c14-ce22-4d70-9644-c725c67cd74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:20.541045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1321/2654] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:20 2022] PUT /identity/v3/projects/8393d07980604e158f8ef4d6de702734/users/172c871291504ff19d70d020ec6c8cce/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:20.546668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1334/2655] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:20.551444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f560c3-5457-4bf5-befd-547d5d4159fd admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.554650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81740a6a-b97a-493b-8060-52feac9b5332 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.554650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81740a6a-b97a-493b-8060-52feac9b5332 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.554650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81740a6a-b97a-493b-8060-52feac9b5332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.583737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-81740a6a-b97a-493b-8060-52feac9b5332 None None] MFA Rules not processed for user `172c871291504ff19d70d020ec6c8cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:20.584976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-81740a6a-b97a-493b-8060-52feac9b5332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:20.630597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f560c3-5457-4bf5-befd-547d5d4159fd admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.637677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c31e9e-d581-4dfb-8360-e0264e14a20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.637677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c31e9e-d581-4dfb-8360-e0264e14a20f admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.637677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c31e9e-d581-4dfb-8360-e0264e14a20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.637677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c31e9e-d581-4dfb-8360-e0264e14a20f admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.638114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c31e9e-d581-4dfb-8360-e0264e14a20f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.639919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c31e9e-d581-4dfb-8360-e0264e14a20f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.659846 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1335/2656] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:20 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:20.675300 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81740a6a-b97a-493b-8060-52feac9b5332 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.676790 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-81740a6a-b97a-493b-8060-52feac9b5332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:20.681372 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:20.737296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1322/2657] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:20 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:20.741702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c31e9e-d581-4dfb-8360-e0264e14a20f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.770990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81740a6a-b97a-493b-8060-52feac9b5332 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.809455 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c31e9e-d581-4dfb-8360-e0264e14a20f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.815549 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc91bd6-e764-4bec-aa0f-ad347479a256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.816344 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc91bd6-e764-4bec-aa0f-ad347479a256 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.816625 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc91bd6-e764-4bec-aa0f-ad347479a256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.816958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc91bd6-e764-4bec-aa0f-ad347479a256 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.818241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc91bd6-e764-4bec-aa0f-ad347479a256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.820264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc91bd6-e764-4bec-aa0f-ad347479a256 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.824048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260123ee-f76e-4191-b2f4-7f359e1ad25f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.824650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260123ee-f76e-4191-b2f4-7f359e1ad25f service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.824960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260123ee-f76e-4191-b2f4-7f359e1ad25f service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.825231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260123ee-f76e-4191-b2f4-7f359e1ad25f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.892227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260123ee-f76e-4191-b2f4-7f359e1ad25f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.894019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260123ee-f76e-4191-b2f4-7f359e1ad25f service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.903464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1336/2658] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:20 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:25:20.938961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1323/2659] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:20 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:20.943005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260123ee-f76e-4191-b2f4-7f359e1ad25f service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.954854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260123ee-f76e-4191-b2f4-7f359e1ad25f service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:20.960429 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7a3ab-5eb2-447e-8583-24e3425e4d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:20.961105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7a3ab-5eb2-447e-8583-24e3425e4d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:20.961390 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7a3ab-5eb2-447e-8583-24e3425e4d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:20.961662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7a3ab-5eb2-447e-8583-24e3425e4d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:20.962304 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7a3ab-5eb2-447e-8583-24e3425e4d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:20.963902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7a3ab-5eb2-447e-8583-24e3425e4d2c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:20.975362 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1337/2660] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:20.979902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc91bd6-e764-4bec-aa0f-ad347479a256 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:20.995221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc91bd6-e764-4bec-aa0f-ad347479a256 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:21.001230 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9833dd-ab78-43c9-bee0-06947852b419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:21.002010 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9833dd-ab78-43c9-bee0-06947852b419 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:21.002287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9833dd-ab78-43c9-bee0-06947852b419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:21.002567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9833dd-ab78-43c9-bee0-06947852b419 admin admin] PATH_INFO: `/v3/projects/cf0a499d3b274194af27171462b7c895/users/da7190a780bb4bed9c2a75fb07958f69/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:21.021911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9833dd-ab78-43c9-bee0-06947852b419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0a499d3b274194af27171462b7c895, user_id=da7190a780bb4bed9c2a75fb07958f69, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:21.024696 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9833dd-ab78-43c9-bee0-06947852b419 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:21.109446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1324/2661] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:20 2022] PUT /identity/v3/projects/cf0a499d3b274194af27171462b7c895/users/da7190a780bb4bed9c2a75fb07958f69/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:21.113780 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7a3ab-5eb2-447e-8583-24e3425e4d2c admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:21.124231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7a3ab-5eb2-447e-8583-24e3425e4d2c admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:21.168392 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:21.183540 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de6c3f-793d-48fc-b659-cb3ab55d89db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:21.184151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de6c3f-793d-48fc-b659-cb3ab55d89db admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:21.184434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de6c3f-793d-48fc-b659-cb3ab55d89db admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:21.184711 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de6c3f-793d-48fc-b659-cb3ab55d89db admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:21.185387 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de6c3f-793d-48fc-b659-cb3ab55d89db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:21.186983 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de6c3f-793d-48fc-b659-cb3ab55d89db admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:21.197498 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1338/2662] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:21.201852 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9833dd-ab78-43c9-bee0-06947852b419 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:21.213190 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9833dd-ab78-43c9-bee0-06947852b419 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:21.218314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8497da-a56f-44c0-b2b0-f07c9aaaf1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:21.219009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8497da-a56f-44c0-b2b0-f07c9aaaf1d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:21.219287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8497da-a56f-44c0-b2b0-f07c9aaaf1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:21.219557 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8497da-a56f-44c0-b2b0-f07c9aaaf1d2 admin admin] PATH_INFO: `/v3/projects/cf0a499d3b274194af27171462b7c895/users/da7190a780bb4bed9c2a75fb07958f69/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:21.221737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8497da-a56f-44c0-b2b0-f07c9aaaf1d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0a499d3b274194af27171462b7c895, user_id=da7190a780bb4bed9c2a75fb07958f69, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:21.224445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8497da-a56f-44c0-b2b0-f07c9aaaf1d2 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:21.236535 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1325/2663] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:21 2022] PUT /identity/v3/projects/cf0a499d3b274194af27171462b7c895/users/da7190a780bb4bed9c2a75fb07958f69/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:21.248272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee03c4a-a507-4c38-8d71-89f9fdd0ca9e None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:21.248487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee03c4a-a507-4c38-8d71-89f9fdd0ca9e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:21.248784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee03c4a-a507-4c38-8d71-89f9fdd0ca9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:21.279761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-aee03c4a-a507-4c38-8d71-89f9fdd0ca9e None None] MFA Rules not processed for user `da7190a780bb4bed9c2a75fb07958f69`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:21.281023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-aee03c4a-a507-4c38-8d71-89f9fdd0ca9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:21.354669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1339/2664] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:21.368397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8497da-a56f-44c0-b2b0-f07c9aaaf1d2 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:21.379522 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:21.426246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8497da-a56f-44c0-b2b0-f07c9aaaf1d2 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:21.430149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3f5d7-6660-4364-acbe-eefedc9fe3db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:21.430827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3f5d7-6660-4364-acbe-eefedc9fe3db service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:21.431106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3f5d7-6660-4364-acbe-eefedc9fe3db service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:21.431505 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3f5d7-6660-4364-acbe-eefedc9fe3db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:21.442322 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3f5d7-6660-4364-acbe-eefedc9fe3db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:21.444569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3f5d7-6660-4364-acbe-eefedc9fe3db service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:21.453945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1326/2665] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:21.732360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee03c4a-a507-4c38-8d71-89f9fdd0ca9e None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:21.743223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee03c4a-a507-4c38-8d71-89f9fdd0ca9e None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:21.747129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857059a-dc51-440c-9f44-628683429368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:21.747732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857059a-dc51-440c-9f44-628683429368 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:21.748009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857059a-dc51-440c-9f44-628683429368 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:21.748279 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857059a-dc51-440c-9f44-628683429368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:21.810639 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857059a-dc51-440c-9f44-628683429368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:21.812214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857059a-dc51-440c-9f44-628683429368 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:21.822232 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1340/2666] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:31.568955 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=22 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:31.817871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=71 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:34.546380 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3f5d7-6660-4364-acbe-eefedc9fe3db service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:34.611751 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3f5d7-6660-4364-acbe-eefedc9fe3db service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:34.617350 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:34.617975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:34.618250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:34.618581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] PATH_INFO: `/v3/users/172c871291504ff19d70d020ec6c8cce` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:34.619720 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=172c871291504ff19d70d020ec6c8cce)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:34.621665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:34.839977 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172c871291504ff19d70d020ec6c8cce'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:34.840645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172c871291504ff19d70d020ec6c8cce'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:34.849245 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172c871291504ff19d70d020ec6c8cce'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:34.861302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172c871291504ff19d70d020ec6c8cce'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:34.980090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172c871291504ff19d70d020ec6c8cce'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:34.987161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1327/2667] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:34 2022] DELETE /identity/v3/users/172c871291504ff19d70d020ec6c8cce => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:34.994310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671755e6-9bf1-4d43-a3db-da6b8a96655f None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:34.994310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671755e6-9bf1-4d43-a3db-da6b8a96655f None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:34.994310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671755e6-9bf1-4d43-a3db-da6b8a96655f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:35.012417 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-671755e6-9bf1-4d43-a3db-da6b8a96655f None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:35.013652 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-671755e6-9bf1-4d43-a3db-da6b8a96655f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:35.075743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1341/2668] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:35.088063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:35.089504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:35.148314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993455a-24b2-4ebe-840f-bcf20c410848 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:35.151811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9945b6-22aa-4918-9d4c-5b9eaf0791d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:35.152486 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9945b6-22aa-4918-9d4c-5b9eaf0791d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:35.152770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9945b6-22aa-4918-9d4c-5b9eaf0791d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:35.153064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9945b6-22aa-4918-9d4c-5b9eaf0791d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:35.161786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9945b6-22aa-4918-9d4c-5b9eaf0791d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:35.163199 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9945b6-22aa-4918-9d4c-5b9eaf0791d7 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:35.173174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1328/2669] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:35.703019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671755e6-9bf1-4d43-a3db-da6b8a96655f None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:35.703019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-671755e6-9bf1-4d43-a3db-da6b8a96655f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:35.715234 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671755e6-9bf1-4d43-a3db-da6b8a96655f None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:35.719999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:35.720631 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:35.720937 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:35.721212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] PATH_INFO: `/v3/projects/8393d07980604e158f8ef4d6de702734` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:35.723308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8393d07980604e158f8ef4d6de702734)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:35.724962 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:35.844207 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8393d07980604e158f8ef4d6de702734', 'request_id': 'req-da6e150f-f59d-481d-a640-6d68cf01b0ca'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:35.850488 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8393d07980604e158f8ef4d6de702734', 'request_id': 'req-da6e150f-f59d-481d-a640-6d68cf01b0ca'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:35.856712 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8393d07980604e158f8ef4d6de702734', 'request_id': 'req-da6e150f-f59d-481d-a640-6d68cf01b0ca'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:35.945925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] The token cache is being invalidate because project 8393d07980604e158f8ef4d6de702734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:35.946161 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:35.946780 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-da6e150f-f59d-481d-a640-6d68cf01b0ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:35.948294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1342/2670] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:35 2022] DELETE /identity/v3/projects/8393d07980604e158f8ef4d6de702734 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:35.953209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5069d1-70c4-452e-8d37-35c78dd4311f None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:35.953719 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5069d1-70c4-452e-8d37-35c78dd4311f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:35.953825 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5069d1-70c4-452e-8d37-35c78dd4311f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:35.974191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0b5069d1-70c4-452e-8d37-35c78dd4311f None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:35.975397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0b5069d1-70c4-452e-8d37-35c78dd4311f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:35.993730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:36.037185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1329/2671] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:36.046119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149d90d-4ad0-4806-957f-63ec7ff57cdc None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:36.046617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149d90d-4ad0-4806-957f-63ec7ff57cdc None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:36.046738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149d90d-4ad0-4806-957f-63ec7ff57cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:36.065893 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-c149d90d-4ad0-4806-957f-63ec7ff57cdc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:36.067577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c149d90d-4ad0-4806-957f-63ec7ff57cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:36.075662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1343/2672] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:36.080680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5069d1-70c4-452e-8d37-35c78dd4311f None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:36.091087 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5069d1-70c4-452e-8d37-35c78dd4311f None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:36.096039 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eedaca-0e86-43c4-9260-aae34fb2ccc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:36.096649 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eedaca-0e86-43c4-9260-aae34fb2ccc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:36.096967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eedaca-0e86-43c4-9260-aae34fb2ccc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:36.097246 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eedaca-0e86-43c4-9260-aae34fb2ccc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:36.098209 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eedaca-0e86-43c4-9260-aae34fb2ccc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:25:36.098569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eedaca-0e86-43c4-9260-aae34fb2ccc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:36.100709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eedaca-0e86-43c4-9260-aae34fb2ccc8 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:36.114949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1330/2673] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:25:36.119389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149d90d-4ad0-4806-957f-63ec7ff57cdc None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:36.129521 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149d90d-4ad0-4806-957f-63ec7ff57cdc None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:36.134237 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba7a75-b252-4c45-a6a8-dc0b12830d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:36.134897 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba7a75-b252-4c45-a6a8-dc0b12830d33 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:36.135240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba7a75-b252-4c45-a6a8-dc0b12830d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:36.135595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba7a75-b252-4c45-a6a8-dc0b12830d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:36.136281 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba7a75-b252-4c45-a6a8-dc0b12830d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:36.138240 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba7a75-b252-4c45-a6a8-dc0b12830d33 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:36.233272 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1344/2674] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:36 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:36.237574 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eedaca-0e86-43c4-9260-aae34fb2ccc8 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:36.248277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eedaca-0e86-43c4-9260-aae34fb2ccc8 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:36.308371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa46ec9-751e-4a30-ba62-25ef80690c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:36.311765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa46ec9-751e-4a30-ba62-25ef80690c49 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:36.311765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa46ec9-751e-4a30-ba62-25ef80690c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:36.311765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa46ec9-751e-4a30-ba62-25ef80690c49 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:36.311765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa46ec9-751e-4a30-ba62-25ef80690c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:36.312184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa46ec9-751e-4a30-ba62-25ef80690c49 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:36.419844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1331/2675] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:36 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:36.423908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba7a75-b252-4c45-a6a8-dc0b12830d33 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:36.435031 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba7a75-b252-4c45-a6a8-dc0b12830d33 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:36.439697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b5bdd-5de1-4980-b07b-07402ae69ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:36.440293 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b5bdd-5de1-4980-b07b-07402ae69ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:36.440577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b5bdd-5de1-4980-b07b-07402ae69ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:36.440848 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b5bdd-5de1-4980-b07b-07402ae69ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:36.442375 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b5bdd-5de1-4980-b07b-07402ae69ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:36.443988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b5bdd-5de1-4980-b07b-07402ae69ec4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:36.453983 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1345/2676] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:36.458339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa46ec9-751e-4a30-ba62-25ef80690c49 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:36.468816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa46ec9-751e-4a30-ba62-25ef80690c49 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:36.473674 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4766620-9295-4002-a6aa-1c94155f98b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:36.474309 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766620-9295-4002-a6aa-1c94155f98b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:36.474591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766620-9295-4002-a6aa-1c94155f98b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:36.474871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766620-9295-4002-a6aa-1c94155f98b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:36.475538 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4766620-9295-4002-a6aa-1c94155f98b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:36.477170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4766620-9295-4002-a6aa-1c94155f98b9 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:36.489294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1332/2677] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:36.493542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b5bdd-5de1-4980-b07b-07402ae69ec4 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:36.504308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b5bdd-5de1-4980-b07b-07402ae69ec4 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:36.509332 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65772f-6b06-49cc-88cc-5cd4d9a9af97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:36.509987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65772f-6b06-49cc-88cc-5cd4d9a9af97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:36.510261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65772f-6b06-49cc-88cc-5cd4d9a9af97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:36.510534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65772f-6b06-49cc-88cc-5cd4d9a9af97 admin admin] PATH_INFO: `/v3/projects/e9980d2c52934ca38dfb4d3dfbf9700b/users/8744189102a44d47a3b994bfe49fd8e1/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:36.525439 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65772f-6b06-49cc-88cc-5cd4d9a9af97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9980d2c52934ca38dfb4d3dfbf9700b, user_id=8744189102a44d47a3b994bfe49fd8e1, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:36.528127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65772f-6b06-49cc-88cc-5cd4d9a9af97 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:36.637022 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1346/2678] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:36 2022] PUT /identity/v3/projects/e9980d2c52934ca38dfb4d3dfbf9700b/users/8744189102a44d47a3b994bfe49fd8e1/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:36.642125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a17062-5d5d-44f2-83e9-f6d39381c793 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:36.642410 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a17062-5d5d-44f2-83e9-f6d39381c793 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:36.642685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a17062-5d5d-44f2-83e9-f6d39381c793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:36.661851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-c9a17062-5d5d-44f2-83e9-f6d39381c793 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:36.663028 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c9a17062-5d5d-44f2-83e9-f6d39381c793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:36.723353 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1333/2679] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:36.733724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65772f-6b06-49cc-88cc-5cd4d9a9af97 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:36.735383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-4e65772f-6b06-49cc-88cc-5cd4d9a9af97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:36.758109 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:36.793453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65772f-6b06-49cc-88cc-5cd4d9a9af97 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:36.797804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9da08d-5df1-44aa-9c91-075f22de51f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:36.798431 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9da08d-5df1-44aa-9c91-075f22de51f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:36.798709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9da08d-5df1-44aa-9c91-075f22de51f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:36.798981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9da08d-5df1-44aa-9c91-075f22de51f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:36.808770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9da08d-5df1-44aa-9c91-075f22de51f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:36.810217 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9da08d-5df1-44aa-9c91-075f22de51f1 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:36.819748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1347/2680] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:40.039801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a17062-5d5d-44f2-83e9-f6d39381c793 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:40.042090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c9a17062-5d5d-44f2-83e9-f6d39381c793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:40.057300 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a17062-5d5d-44f2-83e9-f6d39381c793 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:40.063243 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:40.064105 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:40.064383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:40.064703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] PATH_INFO: `/v3/users/638fe693e1a440588bcd3a546e93ec77` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:40.066059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=638fe693e1a440588bcd3a546e93ec77)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:40.076295 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:40.241891 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:40.439942 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '638fe693e1a440588bcd3a546e93ec77'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:40.440594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '638fe693e1a440588bcd3a546e93ec77'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:40.448125 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '638fe693e1a440588bcd3a546e93ec77'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:40.460784 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '638fe693e1a440588bcd3a546e93ec77'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:40.636507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '638fe693e1a440588bcd3a546e93ec77'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:40.645863 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1334/2681] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:40 2022] DELETE /identity/v3/users/638fe693e1a440588bcd3a546e93ec77 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:40.650585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532993-cadf-47e2-924d-3673a9fe04b0 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:40.651151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532993-cadf-47e2-924d-3673a9fe04b0 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:40.651151 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532993-cadf-47e2-924d-3673a9fe04b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:40.670501 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-06532993-cadf-47e2-924d-3673a9fe04b0 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:40.671688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-06532993-cadf-47e2-924d-3673a9fe04b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:40.736345 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1348/2682] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:40.747073 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:40.748764 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:40.821498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162ae24-689a-4f24-a89a-d2332e49a6a8 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:40.825277 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6227-b302-4f25-90af-3413d73d8dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:40.825956 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6227-b302-4f25-90af-3413d73d8dea service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:40.826304 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6227-b302-4f25-90af-3413d73d8dea service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:40.826765 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6227-b302-4f25-90af-3413d73d8dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:40.836536 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c6227-b302-4f25-90af-3413d73d8dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:40.838181 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c6227-b302-4f25-90af-3413d73d8dea service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:40.848358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1335/2683] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:25:41.583276 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06532993-cadf-47e2-924d-3673a9fe04b0 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:41.584997 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-06532993-cadf-47e2-924d-3673a9fe04b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:41.601322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06532993-cadf-47e2-924d-3673a9fe04b0 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:41.607203 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:41.608218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:41.608644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:41.608961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] PATH_INFO: `/v3/projects/4e389a29436c43358c61d8cc7535d1af` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:41.610146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e389a29436c43358c61d8cc7535d1af)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:41.613584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:41.772083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e389a29436c43358c61d8cc7535d1af', 'request_id': 'req-95d5c399-d207-46e2-9b53-bce2e1850651'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:41.778589 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e389a29436c43358c61d8cc7535d1af', 'request_id': 'req-95d5c399-d207-46e2-9b53-bce2e1850651'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:41.786040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e389a29436c43358c61d8cc7535d1af', 'request_id': 'req-95d5c399-d207-46e2-9b53-bce2e1850651'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:41.968558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] The token cache is being invalidate because project 4e389a29436c43358c61d8cc7535d1af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:41.968815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:41.969504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:41.971028 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1349/2684] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:41 2022] DELETE /identity/v3/projects/4e389a29436c43358c61d8cc7535d1af => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:41.975187 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6227-b302-4f25-90af-3413d73d8dea service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:41.985553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6227-b302-4f25-90af-3413d73d8dea service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:42.064739 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a534cc8-0d48-4fb4-9b50-96fe6f688f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:42.065675 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a534cc8-0d48-4fb4-9b50-96fe6f688f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:42.066681 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a534cc8-0d48-4fb4-9b50-96fe6f688f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:42.067270 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a534cc8-0d48-4fb4-9b50-96fe6f688f4f admin admin] PATH_INFO: `/v3/users/da7190a780bb4bed9c2a75fb07958f69` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:42.068753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a534cc8-0d48-4fb4-9b50-96fe6f688f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=da7190a780bb4bed9c2a75fb07958f69)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:42.071260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a534cc8-0d48-4fb4-9b50-96fe6f688f4f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:42.409900 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:42.412662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-6a534cc8-0d48-4fb4-9b50-96fe6f688f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da7190a780bb4bed9c2a75fb07958f69'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:42.413603 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-6a534cc8-0d48-4fb4-9b50-96fe6f688f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da7190a780bb4bed9c2a75fb07958f69'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:42.422265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-6a534cc8-0d48-4fb4-9b50-96fe6f688f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da7190a780bb4bed9c2a75fb07958f69'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:42.447720 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-6a534cc8-0d48-4fb4-9b50-96fe6f688f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da7190a780bb4bed9c2a75fb07958f69'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:42.532034 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-6a534cc8-0d48-4fb4-9b50-96fe6f688f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da7190a780bb4bed9c2a75fb07958f69'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:42.544678 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1336/2685] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:41 2022] DELETE /identity/v3/users/da7190a780bb4bed9c2a75fb07958f69 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:43.089759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:43.093051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:43.127445 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5c399-d207-46e2-9b53-bce2e1850651 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:43.225004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:43.241586 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:43.247079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:43.247079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:43.247079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] PATH_INFO: `/v3/projects/cf0a499d3b274194af27171462b7c895` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:43.247079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0a499d3b274194af27171462b7c895)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:43.247079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:43.292831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb83ce-0e19-4ea6-bee0-bb119afd8412 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:43.292831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb83ce-0e19-4ea6-bee0-bb119afd8412 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:43.292831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb83ce-0e19-4ea6-bee0-bb119afd8412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:43.326235 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ccdb83ce-0e19-4ea6-bee0-bb119afd8412 None None] MFA Rules not processed for user `8744189102a44d47a3b994bfe49fd8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:43.327450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ccdb83ce-0e19-4ea6-bee0-bb119afd8412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:43.396818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1337/2686] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:43.401065 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb83ce-0e19-4ea6-bee0-bb119afd8412 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:43.402476 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ccdb83ce-0e19-4ea6-bee0-bb119afd8412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:43.423280 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb83ce-0e19-4ea6-bee0-bb119afd8412 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:43.431737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0a499d3b274194af27171462b7c895', 'request_id': 'req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:43.438759 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0a499d3b274194af27171462b7c895', 'request_id': 'req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:43.451380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0a499d3b274194af27171462b7c895', 'request_id': 'req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:43.494059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660826b-1728-4851-b226-9c8b1b2cfd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:43.494689 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660826b-1728-4851-b226-9c8b1b2cfd2f admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:43.495261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660826b-1728-4851-b226-9c8b1b2cfd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:43.495494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660826b-1728-4851-b226-9c8b1b2cfd2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:43.496139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660826b-1728-4851-b226-9c8b1b2cfd2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:43.498158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660826b-1728-4851-b226-9c8b1b2cfd2f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:43.581204 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] The token cache is being invalidate because project cf0a499d3b274194af27171462b7c895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:43.582262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:43.582947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-2ce8880a-9e61-43f1-a28e-52e8f8f4b892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:43.584480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1350/2687] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:43 2022] DELETE /identity/v3/projects/cf0a499d3b274194af27171462b7c895 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:43.589140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f71dcb-adbd-474f-a158-bd3ec10f3a7e None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:43.589420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f71dcb-adbd-474f-a158-bd3ec10f3a7e None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:43.589689 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f71dcb-adbd-474f-a158-bd3ec10f3a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:43.617570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e9f71dcb-adbd-474f-a158-bd3ec10f3a7e None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:43.618829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e9f71dcb-adbd-474f-a158-bd3ec10f3a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:43.622052 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1338/2688] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:43 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:25:43.626409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660826b-1728-4851-b226-9c8b1b2cfd2f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:43.631577 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-f660826b-1728-4851-b226-9c8b1b2cfd2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:43.689743 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1351/2689] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:43.691541 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f71dcb-adbd-474f-a158-bd3ec10f3a7e None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:43.693016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e9f71dcb-adbd-474f-a158-bd3ec10f3a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:43.705096 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f71dcb-adbd-474f-a158-bd3ec10f3a7e None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:43.705850 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660826b-1728-4851-b226-9c8b1b2cfd2f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:43.712612 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:43.712757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86d6e2-ad66-44c9-9e25-21b9c4547ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:43.712890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:43.712890 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:43.713072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86d6e2-ad66-44c9-9e25-21b9c4547ada admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:43.713157 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] PATH_INFO: `/v3/users/101064af8a684b6782f1ebfca51db0eb` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:43.713237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86d6e2-ad66-44c9-9e25-21b9c4547ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:43.713237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86d6e2-ad66-44c9-9e25-21b9c4547ada admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:43.713237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86d6e2-ad66-44c9-9e25-21b9c4547ada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:43.713456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=101064af8a684b6782f1ebfca51db0eb)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:43.714458 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86d6e2-ad66-44c9-9e25-21b9c4547ada admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:43.715175 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:43.838221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1339/2690] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:43 2022] POST /identity/v3/users => generated 298 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:25:43.839717 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fec99-350d-48d7-b158-56bc154457b1 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:43.840023 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fec99-350d-48d7-b158-56bc154457b1 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:43.840303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fec99-350d-48d7-b158-56bc154457b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:43.860072 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b84fec99-350d-48d7-b158-56bc154457b1 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:43.861364 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b84fec99-350d-48d7-b158-56bc154457b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:43.873538 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1340/2691] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:43.874393 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fec99-350d-48d7-b158-56bc154457b1 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:43.885093 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fec99-350d-48d7-b158-56bc154457b1 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:43.890402 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9aa37-4f33-4b5e-bc3a-5600c0c7e966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:43.891041 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9aa37-4f33-4b5e-bc3a-5600c0c7e966 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:43.891320 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9aa37-4f33-4b5e-bc3a-5600c0c7e966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:43.891587 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9aa37-4f33-4b5e-bc3a-5600c0c7e966 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:43.892228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9aa37-4f33-4b5e-bc3a-5600c0c7e966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:43.893871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9aa37-4f33-4b5e-bc3a-5600c0c7e966 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:43.904628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1341/2692] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:43.908780 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9aa37-4f33-4b5e-bc3a-5600c0c7e966 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:43.915248 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9aa37-4f33-4b5e-bc3a-5600c0c7e966 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:43.919848 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fabd9-9499-4da6-a04e-4826488e4f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:43.920405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fabd9-9499-4da6-a04e-4826488e4f43 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:43.920680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fabd9-9499-4da6-a04e-4826488e4f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:43.920992 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fabd9-9499-4da6-a04e-4826488e4f43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:43.921617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fabd9-9499-4da6-a04e-4826488e4f43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:25:43.921911 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fabd9-9499-4da6-a04e-4826488e4f43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:43.923473 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fabd9-9499-4da6-a04e-4826488e4f43 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:43.937740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1342/2693] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:25:43.938632 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fabd9-9499-4da6-a04e-4826488e4f43 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:43.949704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fabd9-9499-4da6-a04e-4826488e4f43 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:43.955802 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e680a-d412-48c8-9e04-bcb1c8566cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:43.956509 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e680a-d412-48c8-9e04-bcb1c8566cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:43.956997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e680a-d412-48c8-9e04-bcb1c8566cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:43.957211 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e680a-d412-48c8-9e04-bcb1c8566cd5 admin admin] PATH_INFO: `/v3/projects/a1d84b385b5e41d38f686196688fb8ba/users/c79287b96d904360ba51048977e2166e/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:43.973947 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e680a-d412-48c8-9e04-bcb1c8566cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d84b385b5e41d38f686196688fb8ba, user_id=c79287b96d904360ba51048977e2166e, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:43.976615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e680a-d412-48c8-9e04-bcb1c8566cd5 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:44.022725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101064af8a684b6782f1ebfca51db0eb'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:44.023410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101064af8a684b6782f1ebfca51db0eb'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:44.031952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101064af8a684b6782f1ebfca51db0eb'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:44.032639 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101064af8a684b6782f1ebfca51db0eb'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:44.102097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1343/2694] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:43 2022] PUT /identity/v3/projects/a1d84b385b5e41d38f686196688fb8ba/users/c79287b96d904360ba51048977e2166e/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:44.102097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e680a-d412-48c8-9e04-bcb1c8566cd5 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:44.102097 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5e5e680a-d412-48c8-9e04-bcb1c8566cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:44.122960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101064af8a684b6782f1ebfca51db0eb'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:44.132662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:44.145294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1352/2695] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:43 2022] DELETE /identity/v3/users/101064af8a684b6782f1ebfca51db0eb => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:44.145294 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:44.146143 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:44.167624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e680a-d412-48c8-9e04-bcb1c8566cd5 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:44.223449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5ef57-8565-4576-bb30-41355aa48b90 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:44.228694 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15bf37-7135-4b1e-86f2-32dfef531609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:44.229340 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15bf37-7135-4b1e-86f2-32dfef531609 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:44.229626 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15bf37-7135-4b1e-86f2-32dfef531609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:44.230133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15bf37-7135-4b1e-86f2-32dfef531609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:44.230647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15bf37-7135-4b1e-86f2-32dfef531609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:44.233241 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15bf37-7135-4b1e-86f2-32dfef531609 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:44.250068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1353/2696] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:44.251659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ade6fb-888d-4d22-b5c3-2382968a8984 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:44.252198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ade6fb-888d-4d22-b5c3-2382968a8984 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:44.252665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ade6fb-888d-4d22-b5c3-2382968a8984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:44.272986 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-14ade6fb-888d-4d22-b5c3-2382968a8984 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:44.274216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-14ade6fb-888d-4d22-b5c3-2382968a8984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:44.283977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1354/2697] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:44.285312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ade6fb-888d-4d22-b5c3-2382968a8984 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:44.295747 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ade6fb-888d-4d22-b5c3-2382968a8984 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:44.298871 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129f619-5ea4-42ab-b7ad-b35f7420fa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:44.299496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129f619-5ea4-42ab-b7ad-b35f7420fa12 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:44.302709 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129f619-5ea4-42ab-b7ad-b35f7420fa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:44.302976 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129f619-5ea4-42ab-b7ad-b35f7420fa12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:44.303565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129f619-5ea4-42ab-b7ad-b35f7420fa12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:44.304738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b787e-dfc9-4742-8c71-3ba3b204e352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:44.305511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b787e-dfc9-4742-8c71-3ba3b204e352 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:44.305680 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129f619-5ea4-42ab-b7ad-b35f7420fa12 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:44.305919 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b787e-dfc9-4742-8c71-3ba3b204e352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:44.306292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b787e-dfc9-4742-8c71-3ba3b204e352 admin admin] PATH_INFO: `/v3/projects/a1d84b385b5e41d38f686196688fb8ba/users/c79287b96d904360ba51048977e2166e/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:44.308342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b787e-dfc9-4742-8c71-3ba3b204e352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d84b385b5e41d38f686196688fb8ba, user_id=c79287b96d904360ba51048977e2166e, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:44.311241 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b787e-dfc9-4742-8c71-3ba3b204e352 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:44.327323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1355/2698] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:44 2022] PUT /identity/v3/projects/a1d84b385b5e41d38f686196688fb8ba/users/c79287b96d904360ba51048977e2166e/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:44.327323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b787e-dfc9-4742-8c71-3ba3b204e352 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:44.327323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a38b787e-dfc9-4742-8c71-3ba3b204e352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:44.392763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b787e-dfc9-4742-8c71-3ba3b204e352 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:44.397152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0500f1-6798-4c61-a186-b32814a5a91c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:44.397762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0500f1-6798-4c61-a186-b32814a5a91c service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:44.398042 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0500f1-6798-4c61-a186-b32814a5a91c service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:44.398312 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0500f1-6798-4c61-a186-b32814a5a91c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:44.432561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1344/2699] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:44 2022] POST /identity/v3/projects => generated 377 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:44.433707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129f619-5ea4-42ab-b7ad-b35f7420fa12 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:44.450814 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:44.469552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0500f1-6798-4c61-a186-b32814a5a91c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:44.471196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0500f1-6798-4c61-a186-b32814a5a91c service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:44.487249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1356/2700] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:44.520798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129f619-5ea4-42ab-b7ad-b35f7420fa12 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:44.525770 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb7cd0-773e-4469-976d-49b0a3e9a381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:44.526702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7cd0-773e-4469-976d-49b0a3e9a381 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:44.526822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7cd0-773e-4469-976d-49b0a3e9a381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:44.527703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7cd0-773e-4469-976d-49b0a3e9a381 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:44.527703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb7cd0-773e-4469-976d-49b0a3e9a381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:44.529823 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb7cd0-773e-4469-976d-49b0a3e9a381 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:44.626952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1345/2701] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:44 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:44.631128 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0500f1-6798-4c61-a186-b32814a5a91c service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:44.641361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0500f1-6798-4c61-a186-b32814a5a91c service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:44.646563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582367e-8c7e-42c1-959f-987c46d83ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:44.647181 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582367e-8c7e-42c1-959f-987c46d83ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:44.647463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582367e-8c7e-42c1-959f-987c46d83ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:44.647736 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582367e-8c7e-42c1-959f-987c46d83ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:44.648380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582367e-8c7e-42c1-959f-987c46d83ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:44.650050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582367e-8c7e-42c1-959f-987c46d83ea3 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:44.660770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1357/2702] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:44.664819 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb7cd0-773e-4469-976d-49b0a3e9a381 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:44.675589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb7cd0-773e-4469-976d-49b0a3e9a381 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:44.680396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2621fa-2c7e-412d-9804-dfa4916a7f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:44.681042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2621fa-2c7e-412d-9804-dfa4916a7f9c admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:44.681319 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2621fa-2c7e-412d-9804-dfa4916a7f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:44.681593 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2621fa-2c7e-412d-9804-dfa4916a7f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:44.682233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2621fa-2c7e-412d-9804-dfa4916a7f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:44.683825 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2621fa-2c7e-412d-9804-dfa4916a7f9c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:44.694583 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1346/2703] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:44.698804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582367e-8c7e-42c1-959f-987c46d83ea3 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:44.709876 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582367e-8c7e-42c1-959f-987c46d83ea3 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:44.715716 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18dedb-11bd-4447-b91a-7b67fa0b9328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:44.716379 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18dedb-11bd-4447-b91a-7b67fa0b9328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:44.716661 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18dedb-11bd-4447-b91a-7b67fa0b9328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:44.716976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18dedb-11bd-4447-b91a-7b67fa0b9328 admin admin] PATH_INFO: `/v3/projects/0c6d521a185441fa83fa4097caa5e07d/users/a2f6c479e905496a977e30b38a3eb7fd/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:44.734215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18dedb-11bd-4447-b91a-7b67fa0b9328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6d521a185441fa83fa4097caa5e07d, user_id=a2f6c479e905496a977e30b38a3eb7fd, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:44.736885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18dedb-11bd-4447-b91a-7b67fa0b9328 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:44.858267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1358/2704] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:44 2022] PUT /identity/v3/projects/0c6d521a185441fa83fa4097caa5e07d/users/a2f6c479e905496a977e30b38a3eb7fd/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:44.869896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd658b-6938-4951-933c-173e48abcd6f None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:44.871525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd658b-6938-4951-933c-173e48abcd6f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:44.871525 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd658b-6938-4951-933c-173e48abcd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:44.903262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-6edd658b-6938-4951-933c-173e48abcd6f None None] MFA Rules not processed for user `a2f6c479e905496a977e30b38a3eb7fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:44.904895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6edd658b-6938-4951-933c-173e48abcd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:44.918015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:45.000821 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1347/2705] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:45.002376 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18dedb-11bd-4447-b91a-7b67fa0b9328 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:45.088307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18dedb-11bd-4447-b91a-7b67fa0b9328 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:45.095662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fde9e3-0665-4c06-954c-9d7d1bbe99f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:45.096383 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fde9e3-0665-4c06-954c-9d7d1bbe99f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.096755 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fde9e3-0665-4c06-954c-9d7d1bbe99f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.097348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fde9e3-0665-4c06-954c-9d7d1bbe99f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.098290 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fde9e3-0665-4c06-954c-9d7d1bbe99f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:45.101485 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fde9e3-0665-4c06-954c-9d7d1bbe99f0 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:45.243641 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1359/2706] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:45 2022] POST /identity/v3/projects => generated 379 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:45.248763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd658b-6938-4951-933c-173e48abcd6f None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:45.276276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd658b-6938-4951-933c-173e48abcd6f None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:45.315758 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fde9e3-0665-4c06-954c-9d7d1bbe99f0 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:45.318490 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-61fde9e3-0665-4c06-954c-9d7d1bbe99f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:45.393295 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe244db5-b8ca-444a-9094-dabf6905682a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:45.393952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244db5-b8ca-444a-9094-dabf6905682a admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.394229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244db5-b8ca-444a-9094-dabf6905682a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.394530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244db5-b8ca-444a-9094-dabf6905682a admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.395330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe244db5-b8ca-444a-9094-dabf6905682a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:45.398100 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe244db5-b8ca-444a-9094-dabf6905682a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:45.432316 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fde9e3-0665-4c06-954c-9d7d1bbe99f0 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:45.436261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:45.436921 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.437366 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.437639 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] PATH_INFO: `/v3/projects/442594393ab4477bbb588b9e25186998` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.438964 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=442594393ab4477bbb588b9e25186998)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:45.441608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:45.456735 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:45.490444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1348/2707] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:45 2022] POST /identity/v3/users => generated 290 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:45.494332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe244db5-b8ca-444a-9094-dabf6905682a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:45.505398 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe244db5-b8ca-444a-9094-dabf6905682a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:45.511115 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac408b-c60f-458c-9ecd-26b9b3f5b8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:45.511729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac408b-c60f-458c-9ecd-26b9b3f5b8bf admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.512009 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac408b-c60f-458c-9ecd-26b9b3f5b8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.512282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac408b-c60f-458c-9ecd-26b9b3f5b8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.512991 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac408b-c60f-458c-9ecd-26b9b3f5b8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:45.514595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac408b-c60f-458c-9ecd-26b9b3f5b8bf admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:45.525130 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1349/2708] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:45.529092 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac408b-c60f-458c-9ecd-26b9b3f5b8bf admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:45.539816 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac408b-c60f-458c-9ecd-26b9b3f5b8bf admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:45.544448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a093ba-03d6-466c-8df5-59b3ab7f7f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:45.545507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a093ba-03d6-466c-8df5-59b3ab7f7f47 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.545507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a093ba-03d6-466c-8df5-59b3ab7f7f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.545746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a093ba-03d6-466c-8df5-59b3ab7f7f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.546272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a093ba-03d6-466c-8df5-59b3ab7f7f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:45.548205 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a093ba-03d6-466c-8df5-59b3ab7f7f47 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:45.561182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1350/2709] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:25:45.565408 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a093ba-03d6-466c-8df5-59b3ab7f7f47 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:45.573505 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442594393ab4477bbb588b9e25186998', 'request_id': 'req-86e6adf8-c42c-4593-a01e-7b74653514a0'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:45.575951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a093ba-03d6-466c-8df5-59b3ab7f7f47 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:45.579449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442594393ab4477bbb588b9e25186998', 'request_id': 'req-86e6adf8-c42c-4593-a01e-7b74653514a0'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:45.587233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442594393ab4477bbb588b9e25186998', 'request_id': 'req-86e6adf8-c42c-4593-a01e-7b74653514a0'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:45.634773 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:45.651488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ca783-f32b-4d15-990d-aaebcc9cf311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:45.652241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ca783-f32b-4d15-990d-aaebcc9cf311 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.658599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ca783-f32b-4d15-990d-aaebcc9cf311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.658599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ca783-f32b-4d15-990d-aaebcc9cf311 admin admin] PATH_INFO: `/v3/projects/f122dfe2c4964bc8bf3dfbaca606aeae/users/588153d9ec20447ab3db133a04d6fda9/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.671728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ca783-f32b-4d15-990d-aaebcc9cf311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f122dfe2c4964bc8bf3dfbaca606aeae, user_id=588153d9ec20447ab3db133a04d6fda9, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:45.676021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ca783-f32b-4d15-990d-aaebcc9cf311 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:45.688342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] The token cache is being invalidate because project 442594393ab4477bbb588b9e25186998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:45.688342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:45.688342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-86e6adf8-c42c-4593-a01e-7b74653514a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:45.690333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1360/2710] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:45 2022] DELETE /identity/v3/projects/442594393ab4477bbb588b9e25186998 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:45.695742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149663a-debf-4895-b6b1-54b5914f34c9 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.695742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149663a-debf-4895-b6b1-54b5914f34c9 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.696136 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149663a-debf-4895-b6b1-54b5914f34c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.716869 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-2149663a-debf-4895-b6b1-54b5914f34c9 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:45.718466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2149663a-debf-4895-b6b1-54b5914f34c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:45.728688 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1361/2711] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:45.737000 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e97757-6c94-498a-96f0-6778d7e62dcd None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.738822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e97757-6c94-498a-96f0-6778d7e62dcd None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.738822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e97757-6c94-498a-96f0-6778d7e62dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.759344 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-04e97757-6c94-498a-96f0-6778d7e62dcd None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:45.763360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-04e97757-6c94-498a-96f0-6778d7e62dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:45.764051 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1351/2712] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:45 2022] PUT /identity/v3/projects/f122dfe2c4964bc8bf3dfbaca606aeae/users/588153d9ec20447ab3db133a04d6fda9/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:45.774722 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63c580-9b86-4548-be68-ff2a7d2417c9 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.775016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63c580-9b86-4548-be68-ff2a7d2417c9 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.775430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63c580-9b86-4548-be68-ff2a7d2417c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.823913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-8a63c580-9b86-4548-be68-ff2a7d2417c9 None None] MFA Rules not processed for user `588153d9ec20447ab3db133a04d6fda9`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:45.825102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8a63c580-9b86-4548-be68-ff2a7d2417c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:45.862380 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1362/2713] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:45.868093 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e97757-6c94-498a-96f0-6778d7e62dcd None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:45.877862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e97757-6c94-498a-96f0-6778d7e62dcd None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:45.887834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a34b5-2c88-4cf1-876e-a7c4fbd4dfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:45.887834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a34b5-2c88-4cf1-876e-a7c4fbd4dfd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.887834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a34b5-2c88-4cf1-876e-a7c4fbd4dfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.887834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a34b5-2c88-4cf1-876e-a7c4fbd4dfd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.888249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a34b5-2c88-4cf1-876e-a7c4fbd4dfd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:25:45.888249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a34b5-2c88-4cf1-876e-a7c4fbd4dfd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:45.888249 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a34b5-2c88-4cf1-876e-a7c4fbd4dfd1 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:45.902773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1363/2714] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:25:45.907044 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a34b5-2c88-4cf1-876e-a7c4fbd4dfd1 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:45.918597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a34b5-2c88-4cf1-876e-a7c4fbd4dfd1 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:45.919951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1352/2715] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:45.928068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c94848-b9e9-4e8c-8e25-3fe7bdc68671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:45.928068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c94848-b9e9-4e8c-8e25-3fe7bdc68671 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.928068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c94848-b9e9-4e8c-8e25-3fe7bdc68671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.928068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c94848-b9e9-4e8c-8e25-3fe7bdc68671 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.929224 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c94848-b9e9-4e8c-8e25-3fe7bdc68671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:45.929448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c94848-b9e9-4e8c-8e25-3fe7bdc68671 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:45.932435 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63c580-9b86-4548-be68-ff2a7d2417c9 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:45.934210 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-8a63c580-9b86-4548-be68-ff2a7d2417c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:45.952011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63c580-9b86-4548-be68-ff2a7d2417c9 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:45.959168 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915757fe-e2f4-4d67-bf98-affa4a0108cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:45.959788 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915757fe-e2f4-4d67-bf98-affa4a0108cf admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:45.960082 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915757fe-e2f4-4d67-bf98-affa4a0108cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:45.960383 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915757fe-e2f4-4d67-bf98-affa4a0108cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:45.961132 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915757fe-e2f4-4d67-bf98-affa4a0108cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:45.963135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915757fe-e2f4-4d67-bf98-affa4a0108cf admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:46.115378 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1364/2716] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:45 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:25:46.116894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c94848-b9e9-4e8c-8e25-3fe7bdc68671 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:46.117061 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1353/2717] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:45 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:46.118069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915757fe-e2f4-4d67-bf98-affa4a0108cf admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:46.203343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c94848-b9e9-4e8c-8e25-3fe7bdc68671 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:46.208695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858f0cf-c589-4878-b337-a24d7930f36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:46.209778 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858f0cf-c589-4878-b337-a24d7930f36e admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.210050 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858f0cf-c589-4878-b337-a24d7930f36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.210319 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858f0cf-c589-4878-b337-a24d7930f36e admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:46.211057 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858f0cf-c589-4878-b337-a24d7930f36e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:46.213706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858f0cf-c589-4878-b337-a24d7930f36e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:46.218470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915757fe-e2f4-4d67-bf98-affa4a0108cf admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:46.226685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755eda1-2856-4d9a-85df-50df10e8ffd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:46.227317 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755eda1-2856-4d9a-85df-50df10e8ffd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.227584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755eda1-2856-4d9a-85df-50df10e8ffd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.227853 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755eda1-2856-4d9a-85df-50df10e8ffd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:46.228963 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755eda1-2856-4d9a-85df-50df10e8ffd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:46.231070 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755eda1-2856-4d9a-85df-50df10e8ffd8 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:46.332945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1365/2718] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:46 2022] POST /identity/v3/users => generated 296 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:46.334218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858f0cf-c589-4878-b337-a24d7930f36e admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:46.336960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1354/2719] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:46 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:46.341651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755eda1-2856-4d9a-85df-50df10e8ffd8 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:46.354934 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858f0cf-c589-4878-b337-a24d7930f36e admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:46.358653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755eda1-2856-4d9a-85df-50df10e8ffd8 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:46.360936 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a75b2-a85f-473d-9c45-77f90bdeb1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:46.362004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a75b2-a85f-473d-9c45-77f90bdeb1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.363230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a75b2-a85f-473d-9c45-77f90bdeb1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.363639 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a75b2-a85f-473d-9c45-77f90bdeb1c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:46.364503 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a75b2-a85f-473d-9c45-77f90bdeb1c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:46.364841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732585b8-b331-4733-a24e-0d782fb0e660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:46.366184 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732585b8-b331-4733-a24e-0d782fb0e660 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.366303 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a75b2-a85f-473d-9c45-77f90bdeb1c3 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:46.366483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732585b8-b331-4733-a24e-0d782fb0e660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.366734 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732585b8-b331-4733-a24e-0d782fb0e660 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:46.367400 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732585b8-b331-4733-a24e-0d782fb0e660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:46.369185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732585b8-b331-4733-a24e-0d782fb0e660 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:46.378725 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1366/2720] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:25:46.382802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a75b2-a85f-473d-9c45-77f90bdeb1c3 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:46.387173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1355/2721] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:25:46.391690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732585b8-b331-4733-a24e-0d782fb0e660 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:46.399715 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a75b2-a85f-473d-9c45-77f90bdeb1c3 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:46.402862 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732585b8-b331-4733-a24e-0d782fb0e660 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:46.406335 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a1ceb-d1ef-4398-8f5b-208931bb7d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:46.407865 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1ceb-d1ef-4398-8f5b-208931bb7d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.408207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7c0a1-e478-4b17-a57a-f4e7a23044ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:46.408622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1ceb-d1ef-4398-8f5b-208931bb7d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.408841 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7c0a1-e478-4b17-a57a-f4e7a23044ac admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.409204 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7c0a1-e478-4b17-a57a-f4e7a23044ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.410442 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7c0a1-e478-4b17-a57a-f4e7a23044ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:46.410741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1ceb-d1ef-4398-8f5b-208931bb7d4f admin admin] PATH_INFO: `/v3/projects/d990954b1bae4e5bb73db98884b5fcf8/users/811e990779994ae2b1dc1f827288d352/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:46.411584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7c0a1-e478-4b17-a57a-f4e7a23044ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:46.413165 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:46.415619 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7c0a1-e478-4b17-a57a-f4e7a23044ac admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:46.434617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1356/2722] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:25:46.435685 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7c0a1-e478-4b17-a57a-f4e7a23044ac admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:46.436531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a1ceb-d1ef-4398-8f5b-208931bb7d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d990954b1bae4e5bb73db98884b5fcf8, user_id=811e990779994ae2b1dc1f827288d352, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:46.439650 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a1ceb-d1ef-4398-8f5b-208931bb7d4f admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:46.460612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7c0a1-e478-4b17-a57a-f4e7a23044ac admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:46.467375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fab1f-0fa3-4869-8041-e7775df56a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:46.467612 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fab1f-0fa3-4869-8041-e7775df56a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.467780 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fab1f-0fa3-4869-8041-e7775df56a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.468092 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fab1f-0fa3-4869-8041-e7775df56a27 admin admin] PATH_INFO: `/v3/projects/7d5efdccb2b64693a0a950c711114f3a/users/82c3bba1fcac455684cc0c8190f8231e/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:46.492496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659fab1f-0fa3-4869-8041-e7775df56a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5efdccb2b64693a0a950c711114f3a, user_id=82c3bba1fcac455684cc0c8190f8231e, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:46.497721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659fab1f-0fa3-4869-8041-e7775df56a27 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:46.528770 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1367/2723] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:46 2022] PUT /identity/v3/projects/d990954b1bae4e5bb73db98884b5fcf8/users/811e990779994ae2b1dc1f827288d352/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:46.533049 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a1ceb-d1ef-4398-8f5b-208931bb7d4f admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:46.569487 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:46.615389 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a1ceb-d1ef-4398-8f5b-208931bb7d4f admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:46.620945 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2af661-845e-4844-b2e0-5adfc2bb9299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:46.621560 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2af661-845e-4844-b2e0-5adfc2bb9299 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.621834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2af661-845e-4844-b2e0-5adfc2bb9299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.622103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2af661-845e-4844-b2e0-5adfc2bb9299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:46.622744 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2af661-845e-4844-b2e0-5adfc2bb9299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:46.624388 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2af661-845e-4844-b2e0-5adfc2bb9299 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:46.636889 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1368/2724] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:46.641068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2af661-845e-4844-b2e0-5adfc2bb9299 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:46.642170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1357/2725] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:46 2022] PUT /identity/v3/projects/7d5efdccb2b64693a0a950c711114f3a/users/82c3bba1fcac455684cc0c8190f8231e/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:46.643665 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bd534-15e6-4d00-a726-1b920e54f188 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.644480 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bd534-15e6-4d00-a726-1b920e54f188 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.644748 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bd534-15e6-4d00-a726-1b920e54f188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:46.652653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2af661-845e-4844-b2e0-5adfc2bb9299 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:46.665148 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b83bd534-15e6-4d00-a726-1b920e54f188 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:46.666387 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b83bd534-15e6-4d00-a726-1b920e54f188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:46.733847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94c86b-87e2-477f-8acd-8c80309d3802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:46.738588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94c86b-87e2-477f-8acd-8c80309d3802 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.738588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94c86b-87e2-477f-8acd-8c80309d3802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.738588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94c86b-87e2-477f-8acd-8c80309d3802 admin admin] PATH_INFO: `/v3/projects/d990954b1bae4e5bb73db98884b5fcf8/users/811e990779994ae2b1dc1f827288d352/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:46.738588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94c86b-87e2-477f-8acd-8c80309d3802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d990954b1bae4e5bb73db98884b5fcf8, user_id=811e990779994ae2b1dc1f827288d352, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:46.740216 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94c86b-87e2-477f-8acd-8c80309d3802 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:46.771374 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1369/2726] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:46 2022] PUT /identity/v3/projects/d990954b1bae4e5bb73db98884b5fcf8/users/811e990779994ae2b1dc1f827288d352/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:25:46.773329 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0d25e-a05f-4928-b11b-39998d485d0a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.773627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0d25e-a05f-4928-b11b-39998d485d0a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.773907 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0d25e-a05f-4928-b11b-39998d485d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:46.777999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1358/2727] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:46.784326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bd534-15e6-4d00-a726-1b920e54f188 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:46.793840 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b83bd534-15e6-4d00-a726-1b920e54f188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:46.829241 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-dbf0d25e-a05f-4928-b11b-39998d485d0a None None] MFA Rules not processed for user `811e990779994ae2b1dc1f827288d352`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:46.831162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-dbf0d25e-a05f-4928-b11b-39998d485d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:46.901422 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bd534-15e6-4d00-a726-1b920e54f188 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:46.957551 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:46.992628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1370/2728] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:46.997692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83d1b9-d1c1-444d-8207-16aea0860764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:46.998318 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83d1b9-d1c1-444d-8207-16aea0860764 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:46.998596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83d1b9-d1c1-444d-8207-16aea0860764 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:46.999139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83d1b9-d1c1-444d-8207-16aea0860764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:47.000098 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0d25e-a05f-4928-b11b-39998d485d0a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:47.001525 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-dbf0d25e-a05f-4928-b11b-39998d485d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:47.087747 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0d25e-a05f-4928-b11b-39998d485d0a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:47.094933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4750926-3424-4584-a4c8-e19a8d0e788c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:47.095576 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750926-3424-4584-a4c8-e19a8d0e788c service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:47.095862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750926-3424-4584-a4c8-e19a8d0e788c service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:47.096138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750926-3424-4584-a4c8-e19a8d0e788c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:47.097046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83d1b9-d1c1-444d-8207-16aea0860764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:47.098767 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83d1b9-d1c1-444d-8207-16aea0860764 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:47.113409 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1359/2729] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:47.121805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4750926-3424-4584-a4c8-e19a8d0e788c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:47.123123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4750926-3424-4584-a4c8-e19a8d0e788c service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:47.146834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:47.156754 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1371/2730] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:47.299580 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83d1b9-d1c1-444d-8207-16aea0860764 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:47.301336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4a83d1b9-d1c1-444d-8207-16aea0860764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:47.318872 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83d1b9-d1c1-444d-8207-16aea0860764 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:47.325885 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a0fc5-2f64-4a1c-8991-0d34c4684a2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:47.326586 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a0fc5-2f64-4a1c-8991-0d34c4684a2a service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:47.328757 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a0fc5-2f64-4a1c-8991-0d34c4684a2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:47.329194 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a0fc5-2f64-4a1c-8991-0d34c4684a2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:47.332695 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-765a0fc5-2f64-4a1c-8991-0d34c4684a2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:47.453287 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a0fc5-2f64-4a1c-8991-0d34c4684a2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:47.455264 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a0fc5-2f64-4a1c-8991-0d34c4684a2a service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:47.469968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1360/2731] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:47.797727 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:49.208509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aefd2e-865b-46e4-bf35-050536555a15 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:49.209218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aefd2e-865b-46e4-bf35-050536555a15 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:49.209218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aefd2e-865b-46e4-bf35-050536555a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:49.292260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e6aefd2e-865b-46e4-bf35-050536555a15 None None] MFA Rules not processed for user `82c3bba1fcac455684cc0c8190f8231e`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:49.295171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e6aefd2e-865b-46e4-bf35-050536555a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:49.386339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1372/2732] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:49.398792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a0fc5-2f64-4a1c-8991-0d34c4684a2a service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:49.413459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a0fc5-2f64-4a1c-8991-0d34c4684a2a service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:49.422046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653415f2-5eec-419b-82f8-ed10871c8792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:49.423291 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653415f2-5eec-419b-82f8-ed10871c8792 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:49.425825 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653415f2-5eec-419b-82f8-ed10871c8792 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:49.426315 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653415f2-5eec-419b-82f8-ed10871c8792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:49.442321 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653415f2-5eec-419b-82f8-ed10871c8792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:49.445176 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653415f2-5eec-419b-82f8-ed10871c8792 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:49.458602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1361/2733] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:51.482597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46951b-1188-4ad7-ad3a-cc14dfd54585 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:51.483073 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46951b-1188-4ad7-ad3a-cc14dfd54585 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:51.483197 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46951b-1188-4ad7-ad3a-cc14dfd54585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:51.514977 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-ef46951b-1188-4ad7-ad3a-cc14dfd54585 None None] MFA Rules not processed for user `c79287b96d904360ba51048977e2166e`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:51.516264 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ef46951b-1188-4ad7-ad3a-cc14dfd54585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:51.581014 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb19d38-64d0-4921-9b5c-320da0bfc8d3 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:51.582568 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb19d38-64d0-4921-9b5c-320da0bfc8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:51.583137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb19d38-64d0-4921-9b5c-320da0bfc8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:51.601726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1373/2734] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:51.614595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-4cb19d38-64d0-4921-9b5c-320da0bfc8d3 None None] MFA Rules not processed for user `82c3bba1fcac455684cc0c8190f8231e`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:51.616011 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4cb19d38-64d0-4921-9b5c-320da0bfc8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:51.620815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46951b-1188-4ad7-ad3a-cc14dfd54585 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:51.622261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ef46951b-1188-4ad7-ad3a-cc14dfd54585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:51.630502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1362/2735] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:25:51.679983 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:51.694855 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46951b-1188-4ad7-ad3a-cc14dfd54585 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:51.701373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b71d7-06f6-49d5-8784-1fbed03168a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0b124ca267b4a4391557ad1396524cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:51.701999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b71d7-06f6-49d5-8784-1fbed03168a9 service glance] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:51.702373 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b71d7-06f6-49d5-8784-1fbed03168a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:51.702585 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b71d7-06f6-49d5-8784-1fbed03168a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:51.705614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e34b71d7-06f6-49d5-8784-1fbed03168a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:51.788456 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b71d7-06f6-49d5-8784-1fbed03168a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:51.790448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b71d7-06f6-49d5-8784-1fbed03168a9 service glance] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:51.801324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1374/2736] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:25:52.119511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a36723-fa24-4d90-8e61-13171ed413fc None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:52.119511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a36723-fa24-4d90-8e61-13171ed413fc None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:52.119511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a36723-fa24-4d90-8e61-13171ed413fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:52.154313 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-b8a36723-fa24-4d90-8e61-13171ed413fc None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:52.155611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b8a36723-fa24-4d90-8e61-13171ed413fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:52.227049 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:52.245502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1363/2737] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:25:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:52.371915 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b71d7-06f6-49d5-8784-1fbed03168a9 service glance] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:52.392617 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b71d7-06f6-49d5-8784-1fbed03168a9 service glance] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:52.399108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:52.399832 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:52.400068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:52.400434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] PATH_INFO: `/v3/users/a2f6c479e905496a977e30b38a3eb7fd` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:52.401811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f6c479e905496a977e30b38a3eb7fd)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:52.404834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:52.409993 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a36723-fa24-4d90-8e61-13171ed413fc None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:52.412702 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-b8a36723-fa24-4d90-8e61-13171ed413fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:52.533808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a36723-fa24-4d90-8e61-13171ed413fc None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:52.540480 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8ad6f-45ed-4353-8b32-af590e3695c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:52.541208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8ad6f-45ed-4353-8b32-af590e3695c9 service swift] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:52.541563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8ad6f-45ed-4353-8b32-af590e3695c9 service swift] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:52.541755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8ad6f-45ed-4353-8b32-af590e3695c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:52.552622 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8ad6f-45ed-4353-8b32-af590e3695c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:52.553808 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8ad6f-45ed-4353-8b32-af590e3695c9 service swift] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:52.562899 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1364/2738] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:25:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:25:52.633936 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f6c479e905496a977e30b38a3eb7fd'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:52.635299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f6c479e905496a977e30b38a3eb7fd'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:52.647309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f6c479e905496a977e30b38a3eb7fd'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:52.649387 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f6c479e905496a977e30b38a3eb7fd'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:52.745432 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f6c479e905496a977e30b38a3eb7fd'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:52.782946 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1375/2739] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:52 2022] DELETE /identity/v3/users/a2f6c479e905496a977e30b38a3eb7fd => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:52.788329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad654d6-3894-4f1f-8e9b-aaf9e873f25b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:52.788927 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad654d6-3894-4f1f-8e9b-aaf9e873f25b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:52.789306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad654d6-3894-4f1f-8e9b-aaf9e873f25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:52.830156 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-4ad654d6-3894-4f1f-8e9b-aaf9e873f25b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:52.832355 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4ad654d6-3894-4f1f-8e9b-aaf9e873f25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:52.980373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1365/2740] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:52.989174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:52.991133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:53.094200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129477c-61e8-4172-b124-b8e87dbbc6f4 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:53.104259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b4c14-c4a8-4a33-b086-e3c0d9361362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:53.104259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b4c14-c4a8-4a33-b086-e3c0d9361362 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:53.104259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b4c14-c4a8-4a33-b086-e3c0d9361362 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:53.104259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b4c14-c4a8-4a33-b086-e3c0d9361362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:53.126292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b4c14-c4a8-4a33-b086-e3c0d9361362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:53.129500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b4c14-c4a8-4a33-b086-e3c0d9361362 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:53.142699 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1376/2741] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:54.103439 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad654d6-3894-4f1f-8e9b-aaf9e873f25b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:54.107731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4ad654d6-3894-4f1f-8e9b-aaf9e873f25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:54.154909 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad654d6-3894-4f1f-8e9b-aaf9e873f25b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:54.177432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:54.178647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:54.182528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:54.183091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] PATH_INFO: `/v3/projects/0c6d521a185441fa83fa4097caa5e07d` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:54.186633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c6d521a185441fa83fa4097caa5e07d)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:54.191851 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:54.413183 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c6d521a185441fa83fa4097caa5e07d', 'request_id': 'req-2b95e915-853d-430f-ac51-77324d0f3766'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:54.430450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c6d521a185441fa83fa4097caa5e07d', 'request_id': 'req-2b95e915-853d-430f-ac51-77324d0f3766'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:54.511260 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c6d521a185441fa83fa4097caa5e07d', 'request_id': 'req-2b95e915-853d-430f-ac51-77324d0f3766'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:54.540831 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] The token cache is being invalidate because project 0c6d521a185441fa83fa4097caa5e07d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:54.541413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:54.542303 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:54.544332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1366/2742] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:54 2022] DELETE /identity/v3/projects/0c6d521a185441fa83fa4097caa5e07d => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:54.548397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b4c14-c4a8-4a33-b086-e3c0d9361362 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:54.568018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b4c14-c4a8-4a33-b086-e3c0d9361362 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:54.577096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:54.578837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:54.580271 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:54.586775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:54.675472 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:54.676607 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:54.677146 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:54.677566 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] PATH_INFO: `/v3/users/588153d9ec20447ab3db133a04d6fda9` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:54.679733 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=588153d9ec20447ab3db133a04d6fda9)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:54.682270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:54.799336 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95e915-853d-430f-ac51-77324d0f3766 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:54.807749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9245d-869e-4f38-abbb-1d75d3f24bb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:54.808844 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9245d-869e-4f38-abbb-1d75d3f24bb3 service placement] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:54.809712 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9245d-869e-4f38-abbb-1d75d3f24bb3 service placement] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:54.810596 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9245d-869e-4f38-abbb-1d75d3f24bb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:54.813782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-13d9245d-869e-4f38-abbb-1d75d3f24bb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:54.832814 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9245d-869e-4f38-abbb-1d75d3f24bb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:54.834544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9245d-869e-4f38-abbb-1d75d3f24bb3 service placement] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:54.854981 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1367/2743] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:54.960619 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '588153d9ec20447ab3db133a04d6fda9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:54.961765 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '588153d9ec20447ab3db133a04d6fda9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:54.972520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '588153d9ec20447ab3db133a04d6fda9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:54.974425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '588153d9ec20447ab3db133a04d6fda9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:55.075802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '588153d9ec20447ab3db133a04d6fda9'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:55.171986 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1377/2744] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:54 2022] DELETE /identity/v3/users/588153d9ec20447ab3db133a04d6fda9 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:55.492312 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9245d-869e-4f38-abbb-1d75d3f24bb3 service placement] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:55.495139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-13d9245d-869e-4f38-abbb-1d75d3f24bb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:55.622386 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9245d-869e-4f38-abbb-1d75d3f24bb3 service placement] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:55.763961 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607424ba-52a3-49f8-ade1-c6ff0e1ea224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:55.765388 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607424ba-52a3-49f8-ade1-c6ff0e1ea224 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:55.765676 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607424ba-52a3-49f8-ade1-c6ff0e1ea224 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:55.765959 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607424ba-52a3-49f8-ade1-c6ff0e1ea224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:55.773507 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-607424ba-52a3-49f8-ade1-c6ff0e1ea224 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:55.858211 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=100 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:55.884152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607424ba-52a3-49f8-ade1-c6ff0e1ea224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:55.885798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607424ba-52a3-49f8-ade1-c6ff0e1ea224 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:55.899083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1368/2745] 10.222.0.30 () {60 vars in 1396 bytes} [Wed Aug 24 10:25:55 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3246 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:56.006911 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:56.008528 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:56.123779 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa15c72-a023-4c18-8aba-762966f5d0bb admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:56.129981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:56.131567 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:56.132066 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:56.132588 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] PATH_INFO: `/v3/projects/f122dfe2c4964bc8bf3dfbaca606aeae` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:56.134208 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f122dfe2c4964bc8bf3dfbaca606aeae)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:56.136434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:56.256397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:56.279449 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f122dfe2c4964bc8bf3dfbaca606aeae', 'request_id': 'req-7b00a785-806c-49ff-a7e6-6088c1201e66'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:56.295012 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f122dfe2c4964bc8bf3dfbaca606aeae', 'request_id': 'req-7b00a785-806c-49ff-a7e6-6088c1201e66'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:56.304818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f122dfe2c4964bc8bf3dfbaca606aeae', 'request_id': 'req-7b00a785-806c-49ff-a7e6-6088c1201e66'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:56.392531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] The token cache is being invalidate because project f122dfe2c4964bc8bf3dfbaca606aeae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:56.392531 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:56.393254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:56.403891 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1378/2746] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:56 2022] DELETE /identity/v3/projects/f122dfe2c4964bc8bf3dfbaca606aeae => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:56.405339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607424ba-52a3-49f8-ade1-c6ff0e1ea224 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:56.430547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607424ba-52a3-49f8-ade1-c6ff0e1ea224 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:56.527523 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:56.528281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:56.528475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:56.528749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] PATH_INFO: `/v3/users/811e990779994ae2b1dc1f827288d352` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:56.530112 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=811e990779994ae2b1dc1f827288d352)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:56.533042 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:56.748730 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '811e990779994ae2b1dc1f827288d352'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:56.749639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '811e990779994ae2b1dc1f827288d352'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:56.775281 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '811e990779994ae2b1dc1f827288d352'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:56.811389 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '811e990779994ae2b1dc1f827288d352'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:56.992595 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '811e990779994ae2b1dc1f827288d352'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:57.013519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1369/2747] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:25:56 2022] DELETE /identity/v3/users/811e990779994ae2b1dc1f827288d352 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:57.760798 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:57.763529 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:57.867365 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00a785-806c-49ff-a7e6-6088c1201e66 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:57.939896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776cb0c-7885-4388-8848-60aad4a83b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:57.941322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776cb0c-7885-4388-8848-60aad4a83b13 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:57.941601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776cb0c-7885-4388-8848-60aad4a83b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:57.941922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776cb0c-7885-4388-8848-60aad4a83b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:57.943896 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f776cb0c-7885-4388-8848-60aad4a83b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:58.001721 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:58.004449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:58.066732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776cb0c-7885-4388-8848-60aad4a83b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:58.068053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776cb0c-7885-4388-8848-60aad4a83b13 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:58.088918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1379/2748] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:58.113797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d915aa3-35ba-4a5b-b5f0-2c6402a1dda5 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:58.119198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:58.122910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:58.123366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:58.123746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] PATH_INFO: `/v3/projects/d990954b1bae4e5bb73db98884b5fcf8` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:58.125332 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d990954b1bae4e5bb73db98884b5fcf8)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:58.128433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:58.131293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:58.337179 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd990954b1bae4e5bb73db98884b5fcf8', 'request_id': 'req-df9fe620-0ac3-46c7-bdd3-174602b46450'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:58.356021 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd990954b1bae4e5bb73db98884b5fcf8', 'request_id': 'req-df9fe620-0ac3-46c7-bdd3-174602b46450'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:58.444018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd990954b1bae4e5bb73db98884b5fcf8', 'request_id': 'req-df9fe620-0ac3-46c7-bdd3-174602b46450'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:58.463302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] The token cache is being invalidate because project d990954b1bae4e5bb73db98884b5fcf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:25:58.463859 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:58.472731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-df9fe620-0ac3-46c7-bdd3-174602b46450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:25:58.474801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1370/2749] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:25:58 2022] DELETE /identity/v3/projects/d990954b1bae4e5bb73db98884b5fcf8 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:58.486447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf1ff3-ff53-45f8-9c0c-340935fb385a None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:58.487212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf1ff3-ff53-45f8-9c0c-340935fb385a None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:58.488215 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf1ff3-ff53-45f8-9c0c-340935fb385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:58.562438 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-2abf1ff3-ff53-45f8-9c0c-340935fb385a None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:58.565306 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2abf1ff3-ff53-45f8-9c0c-340935fb385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:58.625861 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:58.676583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1380/2750] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:58.686693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81a472-5c56-4c85-8939-6a5e44e14487 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:58.687240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81a472-5c56-4c85-8939-6a5e44e14487 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:58.687728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81a472-5c56-4c85-8939-6a5e44e14487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:58.718711 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-df81a472-5c56-4c85-8939-6a5e44e14487 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:58.727881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-df81a472-5c56-4c85-8939-6a5e44e14487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:58.749810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1371/2751] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:58.754479 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf1ff3-ff53-45f8-9c0c-340935fb385a None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:58.770983 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf1ff3-ff53-45f8-9c0c-340935fb385a None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:58.776720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f33967-28f0-4236-aed1-6294853a2db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:58.779140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f33967-28f0-4236-aed1-6294853a2db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:58.780411 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f33967-28f0-4236-aed1-6294853a2db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:58.781006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f33967-28f0-4236-aed1-6294853a2db2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:58.796370 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f33967-28f0-4236-aed1-6294853a2db2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:25:58.796370 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f33967-28f0-4236-aed1-6294853a2db2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:58.796370 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f33967-28f0-4236-aed1-6294853a2db2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:58.806908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1381/2752] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:25:58.812628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81a472-5c56-4c85-8939-6a5e44e14487 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:58.825432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81a472-5c56-4c85-8939-6a5e44e14487 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:58.831731 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d6e1c-2b10-42f5-a941-0308b68f9ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:58.832889 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d6e1c-2b10-42f5-a941-0308b68f9ffd admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:58.833334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d6e1c-2b10-42f5-a941-0308b68f9ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:58.833824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d6e1c-2b10-42f5-a941-0308b68f9ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:58.834658 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d6e1c-2b10-42f5-a941-0308b68f9ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:58.836798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d6e1c-2b10-42f5-a941-0308b68f9ffd admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:59.168623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1372/2753] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:58 2022] POST /identity/v3/projects => generated 379 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:59.169368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f33967-28f0-4236-aed1-6294853a2db2 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:59.185317 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f33967-28f0-4236-aed1-6294853a2db2 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:59.283614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26b4bf-e3b4-4fd0-a634-616b85107dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:59.284499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b4bf-e3b4-4fd0-a634-616b85107dba admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:59.284924 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b4bf-e3b4-4fd0-a634-616b85107dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:59.285299 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b4bf-e3b4-4fd0-a634-616b85107dba admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:59.286295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26b4bf-e3b4-4fd0-a634-616b85107dba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:59.288669 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26b4bf-e3b4-4fd0-a634-616b85107dba admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:59.405784 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1382/2754] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:25:59 2022] POST /identity/v3/users => generated 290 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:25:59.410106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d6e1c-2b10-42f5-a941-0308b68f9ffd admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:59.429519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d6e1c-2b10-42f5-a941-0308b68f9ffd admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:59.436333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f9c5-dd91-4cd7-bbf9-6b046ec9f185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:59.436919 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f9c5-dd91-4cd7-bbf9-6b046ec9f185 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:59.437333 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f9c5-dd91-4cd7-bbf9-6b046ec9f185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:59.437582 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f9c5-dd91-4cd7-bbf9-6b046ec9f185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:59.438405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14f9c5-dd91-4cd7-bbf9-6b046ec9f185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:59.440026 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14f9c5-dd91-4cd7-bbf9-6b046ec9f185 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:59.456923 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1373/2755] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:59.460179 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26b4bf-e3b4-4fd0-a634-616b85107dba admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:59.473742 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26b4bf-e3b4-4fd0-a634-616b85107dba admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:59.479982 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fbe39-1e45-4468-867f-6e99183261ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:59.480593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fbe39-1e45-4468-867f-6e99183261ba admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:59.480895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fbe39-1e45-4468-867f-6e99183261ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:59.481635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fbe39-1e45-4468-867f-6e99183261ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:59.482597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947fbe39-1e45-4468-867f-6e99183261ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:59.485731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947fbe39-1e45-4468-867f-6e99183261ba admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:59.512482 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1383/2756] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:25:59.514399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f9c5-dd91-4cd7-bbf9-6b046ec9f185 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:59.529470 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f9c5-dd91-4cd7-bbf9-6b046ec9f185 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:59.540140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85705ac3-d4a8-412d-8881-368e6706624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:59.540140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85705ac3-d4a8-412d-8881-368e6706624f admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:59.540140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85705ac3-d4a8-412d-8881-368e6706624f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:59.540140 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85705ac3-d4a8-412d-8881-368e6706624f admin admin] PATH_INFO: `/v3/projects/c9f851c1c02d46d5aba8de050bf9c7ed/users/58c5850115a64c78a11ffc9ac0ed35f2/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:59.560463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85705ac3-d4a8-412d-8881-368e6706624f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9f851c1c02d46d5aba8de050bf9c7ed, user_id=58c5850115a64c78a11ffc9ac0ed35f2, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:59.565975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85705ac3-d4a8-412d-8881-368e6706624f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:59.697208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1374/2757] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:25:59 2022] PUT /identity/v3/projects/c9f851c1c02d46d5aba8de050bf9c7ed/users/58c5850115a64c78a11ffc9ac0ed35f2/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:25:59.697420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a256734-f89a-4e0c-8df0-49ca0a4df029 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:59.697420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a256734-f89a-4e0c-8df0-49ca0a4df029 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:59.697420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a256734-f89a-4e0c-8df0-49ca0a4df029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:59.741660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-7a256734-f89a-4e0c-8df0-49ca0a4df029 None None] MFA Rules not processed for user `58c5850115a64c78a11ffc9ac0ed35f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:25:59.743458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-7a256734-f89a-4e0c-8df0-49ca0a4df029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:59.905556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1384/2758] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:25:59.910574 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85705ac3-d4a8-412d-8881-368e6706624f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:00.039837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85705ac3-d4a8-412d-8881-368e6706624f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:00.050718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6042002-b21b-4b53-8c5b-0a0b6470b9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:00.051775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6042002-b21b-4b53-8c5b-0a0b6470b9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:00.052149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6042002-b21b-4b53-8c5b-0a0b6470b9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:00.052495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6042002-b21b-4b53-8c5b-0a0b6470b9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:00.053743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6042002-b21b-4b53-8c5b-0a0b6470b9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:00.057282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6042002-b21b-4b53-8c5b-0a0b6470b9d3 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:00.198912 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1375/2759] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:25:59 2022] POST /identity/v3/projects => generated 381 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:26:00.201097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a256734-f89a-4e0c-8df0-49ca0a4df029 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:00.218212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a256734-f89a-4e0c-8df0-49ca0a4df029 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:00.247516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:00.320447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeec48-eec4-41f1-9786-b2ff54783c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:00.321422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeec48-eec4-41f1-9786-b2ff54783c83 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:00.322364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeec48-eec4-41f1-9786-b2ff54783c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:00.322828 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeec48-eec4-41f1-9786-b2ff54783c83 admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:00.323642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aeec48-eec4-41f1-9786-b2ff54783c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:00.326496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aeec48-eec4-41f1-9786-b2ff54783c83 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:00.425433 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1385/2760] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:26:00 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:26:00.430005 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6042002-b21b-4b53-8c5b-0a0b6470b9d3 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:00.445788 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6042002-b21b-4b53-8c5b-0a0b6470b9d3 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:00.454606 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5814c-4dfc-4845-8248-32858589bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:00.455513 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5814c-4dfc-4845-8248-32858589bd29 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:00.456432 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5814c-4dfc-4845-8248-32858589bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:00.456990 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5814c-4dfc-4845-8248-32858589bd29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:00.457945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5814c-4dfc-4845-8248-32858589bd29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:00.460237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5814c-4dfc-4845-8248-32858589bd29 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:00.480524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1376/2761] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:26:00.484905 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeec48-eec4-41f1-9786-b2ff54783c83 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:00.501137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeec48-eec4-41f1-9786-b2ff54783c83 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:00.506854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d7870-f799-4dc7-82c6-741f20baf3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:00.507601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d7870-f799-4dc7-82c6-741f20baf3da admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:00.509043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d7870-f799-4dc7-82c6-741f20baf3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:00.509043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d7870-f799-4dc7-82c6-741f20baf3da admin admin] PATH_INFO: `/v3/projects/cb10417a0a8a46c28cbd87029491696b/users/ff4d0cf0e69c4111af70d666e5ee19b8/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:00.540364 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d7870-f799-4dc7-82c6-741f20baf3da admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb10417a0a8a46c28cbd87029491696b, user_id=ff4d0cf0e69c4111af70d666e5ee19b8, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:00.543466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d7870-f799-4dc7-82c6-741f20baf3da admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:00.657701 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1386/2762] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:26:00 2022] PUT /identity/v3/projects/cb10417a0a8a46c28cbd87029491696b/users/ff4d0cf0e69c4111af70d666e5ee19b8/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:00.661962 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5814c-4dfc-4845-8248-32858589bd29 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:00.668071 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:00.678573 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5814c-4dfc-4845-8248-32858589bd29 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:00.758783 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61522c-3a03-4edb-877e-04d0ee4b591d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:00.759494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61522c-3a03-4edb-877e-04d0ee4b591d admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:00.759870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61522c-3a03-4edb-877e-04d0ee4b591d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:00.760241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61522c-3a03-4edb-877e-04d0ee4b591d admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:00.761218 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61522c-3a03-4edb-877e-04d0ee4b591d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:00.763031 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61522c-3a03-4edb-877e-04d0ee4b591d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:00.779227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1377/2763] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:26:00.783184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d7870-f799-4dc7-82c6-741f20baf3da admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:00.798710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d7870-f799-4dc7-82c6-741f20baf3da admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:00.805775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5e47-e499-44f9-9ab2-91b4dd6b17a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:00.806628 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5e47-e499-44f9-9ab2-91b4dd6b17a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:00.807018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5e47-e499-44f9-9ab2-91b4dd6b17a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:00.807395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5e47-e499-44f9-9ab2-91b4dd6b17a1 admin admin] PATH_INFO: `/v3/projects/cb10417a0a8a46c28cbd87029491696b/users/ff4d0cf0e69c4111af70d666e5ee19b8/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:00.809661 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a5e47-e499-44f9-9ab2-91b4dd6b17a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb10417a0a8a46c28cbd87029491696b, user_id=ff4d0cf0e69c4111af70d666e5ee19b8, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:00.812894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a5e47-e499-44f9-9ab2-91b4dd6b17a1 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:00.835787 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1387/2764] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:26:00 2022] PUT /identity/v3/projects/cb10417a0a8a46c28cbd87029491696b/users/ff4d0cf0e69c4111af70d666e5ee19b8/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:26:00.847870 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4a5f0-706e-4982-be97-bb4d83d77108 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:00.852792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4a5f0-706e-4982-be97-bb4d83d77108 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:00.852792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4a5f0-706e-4982-be97-bb4d83d77108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:00.896811 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-bcc4a5f0-706e-4982-be97-bb4d83d77108 None None] MFA Rules not processed for user `ff4d0cf0e69c4111af70d666e5ee19b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:00.898797 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-bcc4a5f0-706e-4982-be97-bb4d83d77108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:00.999585 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1378/2765] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:01.013092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5e47-e499-44f9-9ab2-91b4dd6b17a1 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:01.014957 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-ce0a5e47-e499-44f9-9ab2-91b4dd6b17a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:01.096120 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5e47-e499-44f9-9ab2-91b4dd6b17a1 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:01.101629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a1c8e-b008-4828-a848-094556f62fd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:01.102399 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a1c8e-b008-4828-a848-094556f62fd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:01.102802 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a1c8e-b008-4828-a848-094556f62fd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:01.103310 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a1c8e-b008-4828-a848-094556f62fd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:01.120169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a1c8e-b008-4828-a848-094556f62fd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:01.120169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a1c8e-b008-4828-a848-094556f62fd0 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:01.129713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1388/2766] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:02.302393 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4a5f0-706e-4982-be97-bb4d83d77108 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:02.448999 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4a5f0-706e-4982-be97-bb4d83d77108 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:02.458849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:02.460202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:02.461017 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:02.461503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] PATH_INFO: `/v3/users/58c5850115a64c78a11ffc9ac0ed35f2` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:02.465504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c5850115a64c78a11ffc9ac0ed35f2)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:02.468570 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:02.599725 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:02.694756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c5850115a64c78a11ffc9ac0ed35f2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:02.694756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c5850115a64c78a11ffc9ac0ed35f2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:02.699233 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c5850115a64c78a11ffc9ac0ed35f2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:02.717200 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c5850115a64c78a11ffc9ac0ed35f2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:02.826450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c5850115a64c78a11ffc9ac0ed35f2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:02.841083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1379/2767] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:26:02 2022] DELETE /identity/v3/users/58c5850115a64c78a11ffc9ac0ed35f2 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:02.841214 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d827b2a-1cb8-4a02-9a80-69a7a3037902 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:02.841665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d827b2a-1cb8-4a02-9a80-69a7a3037902 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:02.841665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d827b2a-1cb8-4a02-9a80-69a7a3037902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:02.868720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-6d827b2a-1cb8-4a02-9a80-69a7a3037902 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:02.870277 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6d827b2a-1cb8-4a02-9a80-69a7a3037902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:02.944795 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1389/2768] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:02.954516 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:02.957365 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:03.038881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1e8d2-24d3-47a3-a650-c9aa3849f798 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:03.046792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dc9dd-a884-4d48-8c6e-811e531469fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:03.047306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dc9dd-a884-4d48-8c6e-811e531469fe service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:03.047591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dc9dd-a884-4d48-8c6e-811e531469fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:03.047945 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dc9dd-a884-4d48-8c6e-811e531469fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:03.086466 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dc9dd-a884-4d48-8c6e-811e531469fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:03.088007 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dc9dd-a884-4d48-8c6e-811e531469fe service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:03.133910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1380/2769] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:04.040653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d827b2a-1cb8-4a02-9a80-69a7a3037902 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:04.058230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6d827b2a-1cb8-4a02-9a80-69a7a3037902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:04.080224 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d827b2a-1cb8-4a02-9a80-69a7a3037902 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:04.093253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:04.093253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:04.093253 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:04.093853 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] PATH_INFO: `/v3/projects/c9f851c1c02d46d5aba8de050bf9c7ed` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:04.095898 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9f851c1c02d46d5aba8de050bf9c7ed)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:04.099327 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:04.119766 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:04.355944 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9f851c1c02d46d5aba8de050bf9c7ed', 'request_id': 'req-0005b714-4e2a-49f3-ab00-8f8bba47ef01'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:04.362578 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9f851c1c02d46d5aba8de050bf9c7ed', 'request_id': 'req-0005b714-4e2a-49f3-ab00-8f8bba47ef01'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:04.374556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9f851c1c02d46d5aba8de050bf9c7ed', 'request_id': 'req-0005b714-4e2a-49f3-ab00-8f8bba47ef01'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:04.547704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] The token cache is being invalidate because project c9f851c1c02d46d5aba8de050bf9c7ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:26:04.547704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:04.548598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:04.550882 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1390/2770] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:26:04 2022] DELETE /identity/v3/projects/c9f851c1c02d46d5aba8de050bf9c7ed => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:04.554976 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dc9dd-a884-4d48-8c6e-811e531469fe service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:04.582467 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dc9dd-a884-4d48-8c6e-811e531469fe service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:04.615611 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:04.618776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:04.690206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65acdc-4c2e-447d-810f-24244481c1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:04.691334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65acdc-4c2e-447d-810f-24244481c1ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:04.691824 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65acdc-4c2e-447d-810f-24244481c1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:04.692376 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65acdc-4c2e-447d-810f-24244481c1ee admin admin] PATH_INFO: `/v3/users/ff4d0cf0e69c4111af70d666e5ee19b8` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:04.694782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65acdc-4c2e-447d-810f-24244481c1ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff4d0cf0e69c4111af70d666e5ee19b8)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:04.698198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65acdc-4c2e-447d-810f-24244481c1ee admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:04.759192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005b714-4e2a-49f3-ab00-8f8bba47ef01 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:04.767443 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffae23-5b26-4303-84ea-a8fadab802ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:04.769015 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffae23-5b26-4303-84ea-a8fadab802ec service placement] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:04.769494 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffae23-5b26-4303-84ea-a8fadab802ec service placement] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:04.770269 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffae23-5b26-4303-84ea-a8fadab802ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:04.773831 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6cffae23-5b26-4303-84ea-a8fadab802ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:04.790866 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cffae23-5b26-4303-84ea-a8fadab802ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:04.792191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cffae23-5b26-4303-84ea-a8fadab802ec service placement] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:04.824632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1391/2771] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:26:05.001288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-4f65acdc-4c2e-447d-810f-24244481c1ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff4d0cf0e69c4111af70d666e5ee19b8'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:05.003135 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-4f65acdc-4c2e-447d-810f-24244481c1ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff4d0cf0e69c4111af70d666e5ee19b8'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:05.005118 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:05.020445 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-4f65acdc-4c2e-447d-810f-24244481c1ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff4d0cf0e69c4111af70d666e5ee19b8'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:05.037422 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-4f65acdc-4c2e-447d-810f-24244481c1ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff4d0cf0e69c4111af70d666e5ee19b8'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:05.149759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-4f65acdc-4c2e-447d-810f-24244481c1ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff4d0cf0e69c4111af70d666e5ee19b8'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:05.161941 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1381/2772] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:26:04 2022] DELETE /identity/v3/users/ff4d0cf0e69c4111af70d666e5ee19b8 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:05.924823 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffae23-5b26-4303-84ea-a8fadab802ec service placement] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:05.926928 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-6cffae23-5b26-4303-84ea-a8fadab802ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:05.959952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffae23-5b26-4303-84ea-a8fadab802ec service placement] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:06.046847 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:06.047805 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:06.048276 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:06.048730 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] PATH_INFO: `/v3/projects/cb10417a0a8a46c28cbd87029491696b` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:06.050987 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb10417a0a8a46c28cbd87029491696b)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:06.053061 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:06.186043 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:06.215254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb10417a0a8a46c28cbd87029491696b', 'request_id': 'req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:06.224154 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb10417a0a8a46c28cbd87029491696b', 'request_id': 'req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:06.231484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb10417a0a8a46c28cbd87029491696b', 'request_id': 'req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:06.403768 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] The token cache is being invalidate because project cb10417a0a8a46c28cbd87029491696b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:26:06.404009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:06.404632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-bd1c4bcf-d651-44b0-b5a7-2119b9c063a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:06.406732 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1392/2773] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:26:05 2022] DELETE /identity/v3/projects/cb10417a0a8a46c28cbd87029491696b => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:06.412792 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778215c2-ee15-464d-b02e-f63f663f4ad3 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:06.413500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778215c2-ee15-464d-b02e-f63f663f4ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:06.413500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778215c2-ee15-464d-b02e-f63f663f4ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:06.434207 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-778215c2-ee15-464d-b02e-f63f663f4ad3 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:06.435098 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-778215c2-ee15-464d-b02e-f63f663f4ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:06.527972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1382/2774] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:06.536680 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee4ff3-af2c-44e4-b8c2-0cce16d2dfcc None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:06.537385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee4ff3-af2c-44e4-b8c2-0cce16d2dfcc None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:06.538007 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee4ff3-af2c-44e4-b8c2-0cce16d2dfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:06.561637 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-53ee4ff3-af2c-44e4-b8c2-0cce16d2dfcc None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:06.563377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-53ee4ff3-af2c-44e4-b8c2-0cce16d2dfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:06.579187 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778215c2-ee15-464d-b02e-f63f663f4ad3 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:06.579978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1393/2775] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:06.591918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778215c2-ee15-464d-b02e-f63f663f4ad3 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:06.597421 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593ef8c-3ede-42ff-a62e-84ddc62d3a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:06.599160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593ef8c-3ede-42ff-a62e-84ddc62d3a8e admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:06.599160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593ef8c-3ede-42ff-a62e-84ddc62d3a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:06.599160 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593ef8c-3ede-42ff-a62e-84ddc62d3a8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:06.599430 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593ef8c-3ede-42ff-a62e-84ddc62d3a8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:26:06.599667 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593ef8c-3ede-42ff-a62e-84ddc62d3a8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:06.601854 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593ef8c-3ede-42ff-a62e-84ddc62d3a8e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:06.615164 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1383/2776] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:26:06.619925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee4ff3-af2c-44e4-b8c2-0cce16d2dfcc None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:06.630138 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee4ff3-af2c-44e4-b8c2-0cce16d2dfcc None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:06.638097 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7fb32-4b78-4029-9a0a-f7615afe0d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:06.638739 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7fb32-4b78-4029-9a0a-f7615afe0d38 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:06.639162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7fb32-4b78-4029-9a0a-f7615afe0d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:06.639448 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7fb32-4b78-4029-9a0a-f7615afe0d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:06.640261 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7fb32-4b78-4029-9a0a-f7615afe0d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:06.645165 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7fb32-4b78-4029-9a0a-f7615afe0d38 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:06.777811 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1394/2777] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:26:06 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:26:06.782228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593ef8c-3ede-42ff-a62e-84ddc62d3a8e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:06.797206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593ef8c-3ede-42ff-a62e-84ddc62d3a8e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:06.898015 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35272dc8-2bd8-4a6a-9580-16d0a904e8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:06.898418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35272dc8-2bd8-4a6a-9580-16d0a904e8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:06.898704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35272dc8-2bd8-4a6a-9580-16d0a904e8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:06.899000 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35272dc8-2bd8-4a6a-9580-16d0a904e8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:06.899611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35272dc8-2bd8-4a6a-9580-16d0a904e8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:06.901675 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35272dc8-2bd8-4a6a-9580-16d0a904e8a0 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:07.103801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1384/2778] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:26:06 2022] POST /identity/v3/users => generated 292 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:26:07.109909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7fb32-4b78-4029-9a0a-f7615afe0d38 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:07.113027 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b43d8-fd63-48d5-8e47-98f8d01f6d87 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:07.113489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b43d8-fd63-48d5-8e47-98f8d01f6d87 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:07.113810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b43d8-fd63-48d5-8e47-98f8d01f6d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:07.128068 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7fb32-4b78-4029-9a0a-f7615afe0d38 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:07.146330 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4218e6-ef1b-4cd1-a2de-2901832806f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:07.146955 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4218e6-ef1b-4cd1-a2de-2901832806f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:07.147223 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4218e6-ef1b-4cd1-a2de-2901832806f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:07.147507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4218e6-ef1b-4cd1-a2de-2901832806f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:07.148140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4218e6-ef1b-4cd1-a2de-2901832806f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:07.149757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4218e6-ef1b-4cd1-a2de-2901832806f5 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:07.158518 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-406b43d8-fd63-48d5-8e47-98f8d01f6d87 None None] MFA Rules not processed for user `8744189102a44d47a3b994bfe49fd8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:07.159763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-406b43d8-fd63-48d5-8e47-98f8d01f6d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:07.191947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1395/2779] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:26:07.193836 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4218e6-ef1b-4cd1-a2de-2901832806f5 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:07.205581 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4218e6-ef1b-4cd1-a2de-2901832806f5 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:07.220506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d517190-af03-4960-b9c9-a0add837af08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:07.221339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d517190-af03-4960-b9c9-a0add837af08 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:07.221904 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d517190-af03-4960-b9c9-a0add837af08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:07.222188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d517190-af03-4960-b9c9-a0add837af08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:07.222973 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d517190-af03-4960-b9c9-a0add837af08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:07.224834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d517190-af03-4960-b9c9-a0add837af08 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:07.246029 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1396/2780] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:26:07.247710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d517190-af03-4960-b9c9-a0add837af08 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:07.255793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:07.262700 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d517190-af03-4960-b9c9-a0add837af08 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:07.273834 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a5074-7230-4db4-a77c-44088d6926ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:07.275356 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a5074-7230-4db4-a77c-44088d6926ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:07.275838 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a5074-7230-4db4-a77c-44088d6926ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:07.276313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a5074-7230-4db4-a77c-44088d6926ea admin admin] PATH_INFO: `/v3/projects/10882f50b4a342eabe3e204dab9528c5/users/470f8f89b1bf45f192efdbd60645c5c8/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:07.289202 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1385/2781] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:07.319618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a5074-7230-4db4-a77c-44088d6926ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=10882f50b4a342eabe3e204dab9528c5, user_id=470f8f89b1bf45f192efdbd60645c5c8, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:07.328683 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a5074-7230-4db4-a77c-44088d6926ea admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:07.433355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1397/2782] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:26:07 2022] PUT /identity/v3/projects/10882f50b4a342eabe3e204dab9528c5/users/470f8f89b1bf45f192efdbd60645c5c8/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:07.444913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbb936-8542-4823-880f-43e207760e7b None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:07.444913 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbb936-8542-4823-880f-43e207760e7b None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:07.449370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbb936-8542-4823-880f-43e207760e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:07.492229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-2ebbb936-8542-4823-880f-43e207760e7b None None] MFA Rules not processed for user `470f8f89b1bf45f192efdbd60645c5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:07.493547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2ebbb936-8542-4823-880f-43e207760e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:07.598803 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1386/2783] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:07.603591 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a5074-7230-4db4-a77c-44088d6926ea admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:07.700903 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a5074-7230-4db4-a77c-44088d6926ea admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:07.705824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad89c65-2433-476d-8480-855e4b8479d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:07.706484 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad89c65-2433-476d-8480-855e4b8479d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:07.706761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad89c65-2433-476d-8480-855e4b8479d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:07.707040 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad89c65-2433-476d-8480-855e4b8479d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:07.707724 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad89c65-2433-476d-8480-855e4b8479d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:07.709698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad89c65-2433-476d-8480-855e4b8479d6 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:07.858824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1398/2784] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:26:07 2022] POST /identity/v3/projects => generated 383 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:26:07.862892 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbb936-8542-4823-880f-43e207760e7b None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:07.875046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbb936-8542-4823-880f-43e207760e7b None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:07.974006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7cb96-6c13-4b16-948b-dbad8dbb6c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:07.974006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7cb96-6c13-4b16-948b-dbad8dbb6c68 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:07.974006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7cb96-6c13-4b16-948b-dbad8dbb6c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:07.974006 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7cb96-6c13-4b16-948b-dbad8dbb6c68 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:07.974628 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7cb96-6c13-4b16-948b-dbad8dbb6c68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:07.976459 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7cb96-6c13-4b16-948b-dbad8dbb6c68 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:08.101275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1387/2785] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:26:07 2022] POST /identity/v3/users => generated 298 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:26:08.105594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad89c65-2433-476d-8480-855e4b8479d6 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:08.117825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad89c65-2433-476d-8480-855e4b8479d6 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:08.122914 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266ad20-c147-4447-90ad-0075932e0022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:08.123563 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266ad20-c147-4447-90ad-0075932e0022 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:08.123845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266ad20-c147-4447-90ad-0075932e0022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:08.124233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266ad20-c147-4447-90ad-0075932e0022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:08.124720 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266ad20-c147-4447-90ad-0075932e0022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:08.126339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266ad20-c147-4447-90ad-0075932e0022 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:08.141163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1399/2786] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:26:08.151545 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7cb96-6c13-4b16-948b-dbad8dbb6c68 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:08.181566 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7cb96-6c13-4b16-948b-dbad8dbb6c68 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:08.187905 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7f1e6-d95b-423c-bed5-f2e22019ae93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:08.188835 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7f1e6-d95b-423c-bed5-f2e22019ae93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:08.202734 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7f1e6-d95b-423c-bed5-f2e22019ae93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:08.202734 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7f1e6-d95b-423c-bed5-f2e22019ae93 admin admin] PATH_INFO: `/v3/projects/d99fcf1f8d41440694f3e3ba6911c917/users/92c8741bf08348b2878dd0486f79b690/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:08.240399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7f1e6-d95b-423c-bed5-f2e22019ae93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d99fcf1f8d41440694f3e3ba6911c917, user_id=92c8741bf08348b2878dd0486f79b690, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:08.243862 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7f1e6-d95b-423c-bed5-f2e22019ae93 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:08.365849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1388/2787] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:26:08 2022] PUT /identity/v3/projects/d99fcf1f8d41440694f3e3ba6911c917/users/92c8741bf08348b2878dd0486f79b690/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:08.370894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266ad20-c147-4447-90ad-0075932e0022 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:08.383754 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266ad20-c147-4447-90ad-0075932e0022 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:08.412716 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:08.472145 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30243768-f6c5-4d67-8ab5-f252229c44e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:08.472835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30243768-f6c5-4d67-8ab5-f252229c44e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:08.473108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30243768-f6c5-4d67-8ab5-f252229c44e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:08.473491 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30243768-f6c5-4d67-8ab5-f252229c44e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:08.474152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30243768-f6c5-4d67-8ab5-f252229c44e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:08.475867 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30243768-f6c5-4d67-8ab5-f252229c44e4 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:08.495599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1400/2788] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:26:08.498091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7f1e6-d95b-423c-bed5-f2e22019ae93 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:08.522384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7f1e6-d95b-423c-bed5-f2e22019ae93 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:08.533633 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b86645-9fea-40bd-b87c-2fe46cec51ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:08.534798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b86645-9fea-40bd-b87c-2fe46cec51ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:08.535418 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b86645-9fea-40bd-b87c-2fe46cec51ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:08.536048 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b86645-9fea-40bd-b87c-2fe46cec51ec admin admin] PATH_INFO: `/v3/projects/d99fcf1f8d41440694f3e3ba6911c917/users/92c8741bf08348b2878dd0486f79b690/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:08.539928 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b86645-9fea-40bd-b87c-2fe46cec51ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d99fcf1f8d41440694f3e3ba6911c917, user_id=92c8741bf08348b2878dd0486f79b690, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:08.545554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b86645-9fea-40bd-b87c-2fe46cec51ec admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:08.562188 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1389/2789] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:26:08 2022] PUT /identity/v3/projects/d99fcf1f8d41440694f3e3ba6911c917/users/92c8741bf08348b2878dd0486f79b690/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:08.579321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af74e22-1df1-4792-9e04-7085060de6a6 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:08.579321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af74e22-1df1-4792-9e04-7085060de6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:08.579321 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af74e22-1df1-4792-9e04-7085060de6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:08.629231 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-9af74e22-1df1-4792-9e04-7085060de6a6 None None] MFA Rules not processed for user `92c8741bf08348b2878dd0486f79b690`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:08.630846 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9af74e22-1df1-4792-9e04-7085060de6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:08.725595 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1401/2790] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:08.739013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b86645-9fea-40bd-b87c-2fe46cec51ec admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:08.741567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-70b86645-9fea-40bd-b87c-2fe46cec51ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:08.809240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:08.842167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b86645-9fea-40bd-b87c-2fe46cec51ec admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:08.845818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d7c87-c93f-4481-ab0c-3d416298d9df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:08.846687 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d7c87-c93f-4481-ab0c-3d416298d9df service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:08.847008 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d7c87-c93f-4481-ab0c-3d416298d9df service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:08.847288 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d7c87-c93f-4481-ab0c-3d416298d9df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:08.921101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d7c87-c93f-4481-ab0c-3d416298d9df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:08.923553 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d7c87-c93f-4481-ab0c-3d416298d9df service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:08.934849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1390/2791] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:09.346224 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af74e22-1df1-4792-9e04-7085060de6a6 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:09.355254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9af74e22-1df1-4792-9e04-7085060de6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:09.450881 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af74e22-1df1-4792-9e04-7085060de6a6 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:09.452925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:09.453568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:09.453850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:09.454127 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] PATH_INFO: `/v3/users/c28281bb90464004a751699a49c61ebf` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:09.455598 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c28281bb90464004a751699a49c61ebf)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:09.457657 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:09.797584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:09.797584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c28281bb90464004a751699a49c61ebf'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:09.797584 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c28281bb90464004a751699a49c61ebf'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:09.803095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c28281bb90464004a751699a49c61ebf'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:09.803829 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c28281bb90464004a751699a49c61ebf'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:09.898247 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c28281bb90464004a751699a49c61ebf'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:09.917918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1402/2792] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:26:09 2022] DELETE /identity/v3/users/c28281bb90464004a751699a49c61ebf => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:10.254306 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d7c87-c93f-4481-ab0c-3d416298d9df service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:10.272954 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d7c87-c93f-4481-ab0c-3d416298d9df service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:10.344293 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:10.345114 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:10.345555 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:10.346136 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] PATH_INFO: `/v3/projects/f2c601f7e6b64f83919d960b601e9331` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:10.348634 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c601f7e6b64f83919d960b601e9331)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:10.352382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:10.507047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c601f7e6b64f83919d960b601e9331', 'request_id': 'req-6410c04f-6eda-4c42-9a62-0071861fc091'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:10.513474 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c601f7e6b64f83919d960b601e9331', 'request_id': 'req-6410c04f-6eda-4c42-9a62-0071861fc091'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:10.585028 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c601f7e6b64f83919d960b601e9331', 'request_id': 'req-6410c04f-6eda-4c42-9a62-0071861fc091'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:10.602449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] The token cache is being invalidate because project f2c601f7e6b64f83919d960b601e9331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:26:10.602740 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:10.603404 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:10.605066 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1391/2793] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:26:10 2022] DELETE /identity/v3/projects/f2c601f7e6b64f83919d960b601e9331 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:10.609355 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:10.610762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:10.624647 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b2a8c-31b4-4e3e-852e-36f7d502849d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:10.684169 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb47cd7-3fc6-4b3e-b9db-27045c7307ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:10.684920 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb47cd7-3fc6-4b3e-b9db-27045c7307ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:10.685233 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb47cd7-3fc6-4b3e-b9db-27045c7307ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:10.687583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb47cd7-3fc6-4b3e-b9db-27045c7307ef admin admin] PATH_INFO: `/v3/users/ee97ab40f8254b6980508c1f7e6b72fa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:10.687583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb47cd7-3fc6-4b3e-b9db-27045c7307ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee97ab40f8254b6980508c1f7e6b72fa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:10.688664 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb47cd7-3fc6-4b3e-b9db-27045c7307ef admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:10.902227 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-cdb47cd7-3fc6-4b3e-b9db-27045c7307ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee97ab40f8254b6980508c1f7e6b72fa'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:10.902872 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-cdb47cd7-3fc6-4b3e-b9db-27045c7307ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee97ab40f8254b6980508c1f7e6b72fa'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:10.911205 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-cdb47cd7-3fc6-4b3e-b9db-27045c7307ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee97ab40f8254b6980508c1f7e6b72fa'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:10.911871 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-cdb47cd7-3fc6-4b3e-b9db-27045c7307ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee97ab40f8254b6980508c1f7e6b72fa'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:11.019709 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-cdb47cd7-3fc6-4b3e-b9db-27045c7307ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee97ab40f8254b6980508c1f7e6b72fa'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:11.038950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1403/2794] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:26:10 2022] DELETE /identity/v3/users/ee97ab40f8254b6980508c1f7e6b72fa => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:11.297022 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:11.298825 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:11.315715 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410c04f-6eda-4c42-9a62-0071861fc091 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:11.326638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:11.378154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:11.379223 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:11.379679 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:11.380178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] PATH_INFO: `/v3/projects/cd71ee4e96a8476e9a5ef9f54052409c` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:11.381395 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd71ee4e96a8476e9a5ef9f54052409c)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:11.383561 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:11.513890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd71ee4e96a8476e9a5ef9f54052409c', 'request_id': 'req-af1d38c3-d7ad-4f3c-a9d0-68767956099c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:11.520141 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd71ee4e96a8476e9a5ef9f54052409c', 'request_id': 'req-af1d38c3-d7ad-4f3c-a9d0-68767956099c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:11.603349 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd71ee4e96a8476e9a5ef9f54052409c', 'request_id': 'req-af1d38c3-d7ad-4f3c-a9d0-68767956099c'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:11.621280 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] The token cache is being invalidate because project cd71ee4e96a8476e9a5ef9f54052409c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:26:11.621581 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:11.622258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:11.640345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1392/2795] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:26:11 2022] DELETE /identity/v3/projects/cd71ee4e96a8476e9a5ef9f54052409c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:12.493729 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cc9f3-67ad-485f-9070-f98c212f55ec None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:12.494278 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cc9f3-67ad-485f-9070-f98c212f55ec None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:12.494402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cc9f3-67ad-485f-9070-f98c212f55ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:12.520960 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-875cc9f3-67ad-485f-9070-f98c212f55ec None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:12.522500 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-875cc9f3-67ad-485f-9070-f98c212f55ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:12.560972 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:12.595167 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1404/2796] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:26:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:12.605408 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:12.607682 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:12.727706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d38c3-d7ad-4f3c-a9d0-68767956099c admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:12.739352 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f4cee-6948-4a14-b3a2-4191740fb902 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:12.739982 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f4cee-6948-4a14-b3a2-4191740fb902 service swift] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:12.740263 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f4cee-6948-4a14-b3a2-4191740fb902 service swift] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:12.740539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f4cee-6948-4a14-b3a2-4191740fb902 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:12.751305 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f4cee-6948-4a14-b3a2-4191740fb902 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:12.753090 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f4cee-6948-4a14-b3a2-4191740fb902 service swift] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:12.762767 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1393/2797] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:26:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:26:14.608131 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cc9f3-67ad-485f-9070-f98c212f55ec None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:14.610230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-875cc9f3-67ad-485f-9070-f98c212f55ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:14.684761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cc9f3-67ad-485f-9070-f98c212f55ec None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:14.688568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2650cf8-b9b1-49b2-a56b-b52b7d1eefe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:14.689359 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2650cf8-b9b1-49b2-a56b-b52b7d1eefe0 service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:14.691718 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2650cf8-b9b1-49b2-a56b-b52b7d1eefe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:14.692119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2650cf8-b9b1-49b2-a56b-b52b7d1eefe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:14.694289 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e2650cf8-b9b1-49b2-a56b-b52b7d1eefe0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:14.781450 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2650cf8-b9b1-49b2-a56b-b52b7d1eefe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:14.782922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2650cf8-b9b1-49b2-a56b-b52b7d1eefe0 service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:14.794281 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1405/2798] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:21.305556 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f4cee-6948-4a14-b3a2-4191740fb902 service swift] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:21.307078 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-1a0f4cee-6948-4a14-b3a2-4191740fb902 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:21.345746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:21.382096 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f4cee-6948-4a14-b3a2-4191740fb902 service swift] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:21.387761 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:21.388414 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:21.388697 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:21.389012 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] PATH_INFO: `/v3/users/470f8f89b1bf45f192efdbd60645c5c8` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:21.390174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=470f8f89b1bf45f192efdbd60645c5c8)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:21.392155 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:21.574405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '470f8f89b1bf45f192efdbd60645c5c8'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:21.575283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '470f8f89b1bf45f192efdbd60645c5c8'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:21.584139 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '470f8f89b1bf45f192efdbd60645c5c8'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:21.598213 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '470f8f89b1bf45f192efdbd60645c5c8'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:21.675630 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '470f8f89b1bf45f192efdbd60645c5c8'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:21.682997 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1394/2799] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:26:21 2022] DELETE /identity/v3/users/470f8f89b1bf45f192efdbd60645c5c8 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:21.687632 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df750735-14c9-4525-b3d9-337c5e3e5fe2 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:21.695726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df750735-14c9-4525-b3d9-337c5e3e5fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:21.695726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df750735-14c9-4525-b3d9-337c5e3e5fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:21.711535 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-df750735-14c9-4525-b3d9-337c5e3e5fe2 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:21.713006 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-df750735-14c9-4525-b3d9-337c5e3e5fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:21.788480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1406/2800] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:21.801265 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:21.802845 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:21.870620 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5d99d-bb5a-4f12-be3e-53189660e951 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:21.874704 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc7536-55d8-467f-8560-a2c98655b1c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:21.875346 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc7536-55d8-467f-8560-a2c98655b1c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:21.875658 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc7536-55d8-467f-8560-a2c98655b1c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:21.875943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc7536-55d8-467f-8560-a2c98655b1c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:21.887858 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc7536-55d8-467f-8560-a2c98655b1c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:21.889427 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc7536-55d8-467f-8560-a2c98655b1c9 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:21.900952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1395/2801] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:22.365149 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df750735-14c9-4525-b3d9-337c5e3e5fe2 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:22.371284 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-df750735-14c9-4525-b3d9-337c5e3e5fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:22.391821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df750735-14c9-4525-b3d9-337c5e3e5fe2 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:22.397457 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:22.397952 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:22.398235 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:22.398515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] PATH_INFO: `/v3/projects/10882f50b4a342eabe3e204dab9528c5` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:22.402397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] RBAC: Authorizing `identity:delete_project(project_id=10882f50b4a342eabe3e204dab9528c5)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:22.402397 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:22.544803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:22.558104 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10882f50b4a342eabe3e204dab9528c5', 'request_id': 'req-77fc316e-328c-4246-8e83-eb8be33c308d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:22.564030 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10882f50b4a342eabe3e204dab9528c5', 'request_id': 'req-77fc316e-328c-4246-8e83-eb8be33c308d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:22.571606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10882f50b4a342eabe3e204dab9528c5', 'request_id': 'req-77fc316e-328c-4246-8e83-eb8be33c308d'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:22.693850 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] The token cache is being invalidate because project 10882f50b4a342eabe3e204dab9528c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:26:22.694535 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:22.695238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:22.696746 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1407/2802] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:26:22 2022] DELETE /identity/v3/projects/10882f50b4a342eabe3e204dab9528c5 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:22.700796 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc7536-55d8-467f-8560-a2c98655b1c9 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:22.716925 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc7536-55d8-467f-8560-a2c98655b1c9 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:22.791251 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:22.800604 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a07f1-835b-47bf-ae9b-75d1d65b480d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:22.801279 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a07f1-835b-47bf-ae9b-75d1d65b480d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:22.801552 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a07f1-835b-47bf-ae9b-75d1d65b480d admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:22.801822 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a07f1-835b-47bf-ae9b-75d1d65b480d admin admin] PATH_INFO: `/v3/users/92c8741bf08348b2878dd0486f79b690` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:22.802946 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a07f1-835b-47bf-ae9b-75d1d65b480d admin admin] RBAC: Authorizing `identity:delete_user(user_id=92c8741bf08348b2878dd0486f79b690)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:22.804943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a07f1-835b-47bf-ae9b-75d1d65b480d admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:22.988403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-216a07f1-835b-47bf-ae9b-75d1d65b480d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92c8741bf08348b2878dd0486f79b690'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:22.989262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-216a07f1-835b-47bf-ae9b-75d1d65b480d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92c8741bf08348b2878dd0486f79b690'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:22.996763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-216a07f1-835b-47bf-ae9b-75d1d65b480d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92c8741bf08348b2878dd0486f79b690'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:23.013302 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-216a07f1-835b-47bf-ae9b-75d1d65b480d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92c8741bf08348b2878dd0486f79b690'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:23.067755 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-216a07f1-835b-47bf-ae9b-75d1d65b480d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92c8741bf08348b2878dd0486f79b690'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:23.077330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1396/2803] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:26:22 2022] DELETE /identity/v3/users/92c8741bf08348b2878dd0486f79b690 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:23.543279 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:23.545862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:23.567017 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc316e-328c-4246-8e83-eb8be33c308d admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:23.671369 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:23.672480 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:23.672761 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:23.673517 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] PATH_INFO: `/v3/projects/d99fcf1f8d41440694f3e3ba6911c917` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:23.674594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d99fcf1f8d41440694f3e3ba6911c917)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:23.677884 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:23.813618 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd99fcf1f8d41440694f3e3ba6911c917', 'request_id': 'req-f6859307-309f-4d15-bba8-be74af369b5b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:23.824515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd99fcf1f8d41440694f3e3ba6911c917', 'request_id': 'req-f6859307-309f-4d15-bba8-be74af369b5b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:23.831803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd99fcf1f8d41440694f3e3ba6911c917', 'request_id': 'req-f6859307-309f-4d15-bba8-be74af369b5b'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:23.927238 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] The token cache is being invalidate because project d99fcf1f8d41440694f3e3ba6911c917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:26:23.930461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:23.930461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-f6859307-309f-4d15-bba8-be74af369b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:23.930461 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1408/2804] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:26:23 2022] DELETE /identity/v3/projects/d99fcf1f8d41440694f3e3ba6911c917 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:23.936197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35126aad-a8ba-4034-8659-86dd654b10a6 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:23.936735 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35126aad-a8ba-4034-8659-86dd654b10a6 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:23.936876 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35126aad-a8ba-4034-8659-86dd654b10a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:23.958865 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-35126aad-a8ba-4034-8659-86dd654b10a6 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:23.960152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-35126aad-a8ba-4034-8659-86dd654b10a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:24.029329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1397/2805] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:24.039665 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0969b0e-29dd-43a7-b57f-bdbe37cc43b2 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:24.039950 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0969b0e-29dd-43a7-b57f-bdbe37cc43b2 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:24.040220 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0969b0e-29dd-43a7-b57f-bdbe37cc43b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:24.061027 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-e0969b0e-29dd-43a7-b57f-bdbe37cc43b2 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:24.062527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e0969b0e-29dd-43a7-b57f-bdbe37cc43b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:24.072007 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1409/2806] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:24.076696 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35126aad-a8ba-4034-8659-86dd654b10a6 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:24.087289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35126aad-a8ba-4034-8659-86dd654b10a6 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:24.093371 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bee58-3e07-4bf6-8438-2e18dd240a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:24.093978 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bee58-3e07-4bf6-8438-2e18dd240a2a admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:24.094256 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bee58-3e07-4bf6-8438-2e18dd240a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:24.094527 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bee58-3e07-4bf6-8438-2e18dd240a2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:24.095250 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1bee58-3e07-4bf6-8438-2e18dd240a2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81930) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:26:24.095485 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1bee58-3e07-4bf6-8438-2e18dd240a2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:24.097488 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1bee58-3e07-4bf6-8438-2e18dd240a2a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:24.109890 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1398/2807] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:26:24.115180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0969b0e-29dd-43a7-b57f-bdbe37cc43b2 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:24.124458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0969b0e-29dd-43a7-b57f-bdbe37cc43b2 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:24.132689 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1ce32-9b17-4297-923c-88c51af5dd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:24.133454 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1ce32-9b17-4297-923c-88c51af5dd95 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:24.133731 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1ce32-9b17-4297-923c-88c51af5dd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:24.134005 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1ce32-9b17-4297-923c-88c51af5dd95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:24.134643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1ce32-9b17-4297-923c-88c51af5dd95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:24.136782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1ce32-9b17-4297-923c-88c51af5dd95 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:24.287821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1410/2808] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:26:24 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:26:24.292232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bee58-3e07-4bf6-8438-2e18dd240a2a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:24.305577 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bee58-3e07-4bf6-8438-2e18dd240a2a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:24.366605 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655aa6f-f355-470e-b13c-8fb92480618e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:24.367219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655aa6f-f355-470e-b13c-8fb92480618e admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:24.367496 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655aa6f-f355-470e-b13c-8fb92480618e admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:24.367763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655aa6f-f355-470e-b13c-8fb92480618e admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:24.368384 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655aa6f-f355-470e-b13c-8fb92480618e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:24.370501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655aa6f-f355-470e-b13c-8fb92480618e admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:24.379960 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:24.489363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1399/2809] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:26:24 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:26:24.493398 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1ce32-9b17-4297-923c-88c51af5dd95 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:24.504808 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1ce32-9b17-4297-923c-88c51af5dd95 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:24.509716 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e887f7b-1f8f-47e8-8a3a-292d83da0674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:24.510311 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e887f7b-1f8f-47e8-8a3a-292d83da0674 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:24.510590 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e887f7b-1f8f-47e8-8a3a-292d83da0674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:24.510863 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e887f7b-1f8f-47e8-8a3a-292d83da0674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:24.511502 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e887f7b-1f8f-47e8-8a3a-292d83da0674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:24.513270 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e887f7b-1f8f-47e8-8a3a-292d83da0674 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:24.524824 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1411/2810] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:26:24.532414 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655aa6f-f355-470e-b13c-8fb92480618e admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:24.543446 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655aa6f-f355-470e-b13c-8fb92480618e admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:24.548211 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a2f67-cc4e-4aaf-9ae0-676bb341ab56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:24.549528 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a2f67-cc4e-4aaf-9ae0-676bb341ab56 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:24.549896 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a2f67-cc4e-4aaf-9ae0-676bb341ab56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:24.550393 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a2f67-cc4e-4aaf-9ae0-676bb341ab56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:24.551060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a2f67-cc4e-4aaf-9ae0-676bb341ab56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:24.552662 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a2f67-cc4e-4aaf-9ae0-676bb341ab56 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:24.563914 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1400/2811] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:26:24.568132 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e887f7b-1f8f-47e8-8a3a-292d83da0674 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:24.578594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e887f7b-1f8f-47e8-8a3a-292d83da0674 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:24.583613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d093e1-71dc-477d-bfc2-e9d582d656c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:24.584255 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d093e1-71dc-477d-bfc2-e9d582d656c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:24.584533 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d093e1-71dc-477d-bfc2-e9d582d656c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:24.584803 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d093e1-71dc-477d-bfc2-e9d582d656c0 admin admin] PATH_INFO: `/v3/projects/2b943b81cb484401becd0bf732a91716/users/425f4b989d34419caef7c6b5c37b69c8/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:24.587994 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:24.601510 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d093e1-71dc-477d-bfc2-e9d582d656c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b943b81cb484401becd0bf732a91716, user_id=425f4b989d34419caef7c6b5c37b69c8, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:24.604342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d093e1-71dc-477d-bfc2-e9d582d656c0 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:24.703583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1412/2812] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:26:24 2022] PUT /identity/v3/projects/2b943b81cb484401becd0bf732a91716/users/425f4b989d34419caef7c6b5c37b69c8/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:24.715340 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f99df-a28e-4125-91ee-2b91f3795a31 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:24.715621 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f99df-a28e-4125-91ee-2b91f3795a31 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:24.715892 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f99df-a28e-4125-91ee-2b91f3795a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:24.747651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-472f99df-a28e-4125-91ee-2b91f3795a31 None None] MFA Rules not processed for user `425f4b989d34419caef7c6b5c37b69c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:24.747651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-472f99df-a28e-4125-91ee-2b91f3795a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:24.818047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1401/2813] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:24.822545 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d093e1-71dc-477d-bfc2-e9d582d656c0 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:24.888853 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d093e1-71dc-477d-bfc2-e9d582d656c0 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:24.894420 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b921600-26a1-4a46-b2e7-dd8bb51a5294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:24.895016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b921600-26a1-4a46-b2e7-dd8bb51a5294 admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:24.895298 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b921600-26a1-4a46-b2e7-dd8bb51a5294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:24.895571 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b921600-26a1-4a46-b2e7-dd8bb51a5294 admin admin] PATH_INFO: `/v3/projects` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:24.896200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b921600-26a1-4a46-b2e7-dd8bb51a5294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:24.898129 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b921600-26a1-4a46-b2e7-dd8bb51a5294 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:25.411076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1413/2814] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:26:24 2022] POST /identity/v3/projects => generated 375 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:26:25.414197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f99df-a28e-4125-91ee-2b91f3795a31 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:25.414197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f99df-a28e-4125-91ee-2b91f3795a31 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:25.414197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85117bb3-78ee-47d2-b9fa-8802535c1d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:25.414197 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85117bb3-78ee-47d2-b9fa-8802535c1d01 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:25.414834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85117bb3-78ee-47d2-b9fa-8802535c1d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:25.414834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85117bb3-78ee-47d2-b9fa-8802535c1d01 admin admin] PATH_INFO: `/v3/users` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:25.414834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85117bb3-78ee-47d2-b9fa-8802535c1d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:25.414834 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85117bb3-78ee-47d2-b9fa-8802535c1d01 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:25.688177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1402/2815] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:26:25 2022] POST /identity/v3/users => generated 294 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:26:25.694371 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b921600-26a1-4a46-b2e7-dd8bb51a5294 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:25.704844 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b921600-26a1-4a46-b2e7-dd8bb51a5294 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:25.709990 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada287e1-2bf0-423c-8ae7-9d7baee56c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:25.710615 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada287e1-2bf0-423c-8ae7-9d7baee56c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:25.710919 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada287e1-2bf0-423c-8ae7-9d7baee56c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:25.711196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada287e1-2bf0-423c-8ae7-9d7baee56c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:25.711837 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada287e1-2bf0-423c-8ae7-9d7baee56c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:25.713513 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada287e1-2bf0-423c-8ae7-9d7baee56c80 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:25.724624 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1414/2816] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:26:25.728832 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85117bb3-78ee-47d2-b9fa-8802535c1d01 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:25.739799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85117bb3-78ee-47d2-b9fa-8802535c1d01 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:25.746339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2305e8-229b-4932-8e5e-00fd210ec25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:25.746339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2305e8-229b-4932-8e5e-00fd210ec25a admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:25.746339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2305e8-229b-4932-8e5e-00fd210ec25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:25.746339 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2305e8-229b-4932-8e5e-00fd210ec25a admin admin] PATH_INFO: `/v3/projects/88332941eb5c4154b87069aa5aec928d/users/74124820abce47ea93bb23b2c797d0ee/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:25.763286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2305e8-229b-4932-8e5e-00fd210ec25a admin admin] RBAC: Authorizing `identity:create_grant(project_id=88332941eb5c4154b87069aa5aec928d, user_id=74124820abce47ea93bb23b2c797d0ee, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:25.765973 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2305e8-229b-4932-8e5e-00fd210ec25a admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:25.835970 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1403/2817] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:26:25 2022] PUT /identity/v3/projects/88332941eb5c4154b87069aa5aec928d/users/74124820abce47ea93bb23b2c797d0ee/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:25.840082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada287e1-2bf0-423c-8ae7-9d7baee56c80 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:25.850885 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada287e1-2bf0-423c-8ae7-9d7baee56c80 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:25.911252 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f1031-4ba2-4bf0-a6b7-2db16f981b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:25.911862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1031-4ba2-4bf0-a6b7-2db16f981b9c admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:25.912147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1031-4ba2-4bf0-a6b7-2db16f981b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:25.912422 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1031-4ba2-4bf0-a6b7-2db16f981b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:25.913123 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f1031-4ba2-4bf0-a6b7-2db16f981b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:25.914757 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f1031-4ba2-4bf0-a6b7-2db16f981b9c admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:25.925302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1415/2818] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:26:25.929931 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2305e8-229b-4932-8e5e-00fd210ec25a admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:25.942018 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2305e8-229b-4932-8e5e-00fd210ec25a admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:25.956733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deabab2-9286-4ea0-9bcd-4a9870d722c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:25.956733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deabab2-9286-4ea0-9bcd-4a9870d722c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:25.956733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deabab2-9286-4ea0-9bcd-4a9870d722c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:25.956733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deabab2-9286-4ea0-9bcd-4a9870d722c0 admin admin] PATH_INFO: `/v3/projects/88332941eb5c4154b87069aa5aec928d/users/74124820abce47ea93bb23b2c797d0ee/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:25.957152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deabab2-9286-4ea0-9bcd-4a9870d722c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88332941eb5c4154b87069aa5aec928d, user_id=74124820abce47ea93bb23b2c797d0ee, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:25.957152 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deabab2-9286-4ea0-9bcd-4a9870d722c0 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:25.964949 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1404/2819] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:26:25 2022] PUT /identity/v3/projects/88332941eb5c4154b87069aa5aec928d/users/74124820abce47ea93bb23b2c797d0ee/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:26:25.975307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7c799-87f9-4512-a8ea-48f0fb318864 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:25.975599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7c799-87f9-4512-a8ea-48f0fb318864 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:25.975906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7c799-87f9-4512-a8ea-48f0fb318864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:26.007038 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-bec7c799-87f9-4512-a8ea-48f0fb318864 None None] MFA Rules not processed for user `74124820abce47ea93bb23b2c797d0ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:26.008302 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-bec7c799-87f9-4512-a8ea-48f0fb318864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:26.055018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:26.079804 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1416/2820] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:26.091407 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deabab2-9286-4ea0-9bcd-4a9870d722c0 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:26.093129 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5deabab2-9286-4ea0-9bcd-4a9870d722c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:26.123491 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:26.156262 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deabab2-9286-4ea0-9bcd-4a9870d722c0 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:26.160670 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db147864-082b-4658-95dd-51ab18e3d740 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:26.161299 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db147864-082b-4658-95dd-51ab18e3d740 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:26.161579 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db147864-082b-4658-95dd-51ab18e3d740 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:26.161856 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db147864-082b-4658-95dd-51ab18e3d740 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:26.217048 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db147864-082b-4658-95dd-51ab18e3d740 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:26.218489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db147864-082b-4658-95dd-51ab18e3d740 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:26.228599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1405/2821] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:26:29.387076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7c799-87f9-4512-a8ea-48f0fb318864 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:29.460713 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7c799-87f9-4512-a8ea-48f0fb318864 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:29.465597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:29.466225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:29.477542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:29.477542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] PATH_INFO: `/v3/users/425f4b989d34419caef7c6b5c37b69c8` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:29.477542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=425f4b989d34419caef7c6b5c37b69c8)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:29.477542 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:29.719053 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425f4b989d34419caef7c6b5c37b69c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:29.719704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425f4b989d34419caef7c6b5c37b69c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:29.728568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425f4b989d34419caef7c6b5c37b69c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:29.729391 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425f4b989d34419caef7c6b5c37b69c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:29.807276 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425f4b989d34419caef7c6b5c37b69c8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:29.828367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1417/2822] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:26:29 2022] DELETE /identity/v3/users/425f4b989d34419caef7c6b5c37b69c8 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:29.833163 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fdc18-6e3f-4ac0-9366-97c24cc6d74c None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:29.833660 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fdc18-6e3f-4ac0-9366-97c24cc6d74c None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:29.835298 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fdc18-6e3f-4ac0-9366-97c24cc6d74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:29.853652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-304fdc18-6e3f-4ac0-9366-97c24cc6d74c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:29.854874 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-304fdc18-6e3f-4ac0-9366-97c24cc6d74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:29.918016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1406/2823] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:29.929105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:29.930530 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:29.991849 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bf188-85b9-4f45-b842-27ab28fff2b7 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:29.995478 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c55eb-df56-478f-8abc-5e63ac73be8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:29.996083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c55eb-df56-478f-8abc-5e63ac73be8d service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:29.996363 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c55eb-df56-478f-8abc-5e63ac73be8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:29.996634 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c55eb-df56-478f-8abc-5e63ac73be8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:30.006333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c55eb-df56-478f-8abc-5e63ac73be8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:30.009200 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c55eb-df56-478f-8abc-5e63ac73be8d service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:30.017976 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1418/2824] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:30.454849 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fdc18-6e3f-4ac0-9366-97c24cc6d74c None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:30.456920 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-304fdc18-6e3f-4ac0-9366-97c24cc6d74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:30.469167 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fdc18-6e3f-4ac0-9366-97c24cc6d74c None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:30.476192 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:30.476657 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:30.477064 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:30.477241 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] PATH_INFO: `/v3/projects/2b943b81cb484401becd0bf732a91716` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:30.478569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b943b81cb484401becd0bf732a91716)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:30.480562 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:30.591571 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b943b81cb484401becd0bf732a91716', 'request_id': 'req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:30.598441 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b943b81cb484401becd0bf732a91716', 'request_id': 'req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:30.682804 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b943b81cb484401becd0bf732a91716', 'request_id': 'req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:30.692651 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:30.697888 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] The token cache is being invalidate because project 2b943b81cb484401becd0bf732a91716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:26:30.698124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:30.698768 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:30.700498 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1407/2825] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:26:30 2022] DELETE /identity/v3/projects/2b943b81cb484401becd0bf732a91716 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:30.704296 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c55eb-df56-478f-8abc-5e63ac73be8d service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:30.710594 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:30.716463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c55eb-df56-478f-8abc-5e63ac73be8d service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:30.781706 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5ef8d-22c3-47a0-b647-ca3ef8722fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:30.782562 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5ef8d-22c3-47a0-b647-ca3ef8722fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:30.782968 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5ef8d-22c3-47a0-b647-ca3ef8722fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:30.783313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5ef8d-22c3-47a0-b647-ca3ef8722fe8 admin admin] PATH_INFO: `/v3/users/74124820abce47ea93bb23b2c797d0ee` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:30.784690 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5ef8d-22c3-47a0-b647-ca3ef8722fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74124820abce47ea93bb23b2c797d0ee)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:30.790552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5ef8d-22c3-47a0-b647-ca3ef8722fe8 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:31.060162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-5cd5ef8d-22c3-47a0-b647-ca3ef8722fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74124820abce47ea93bb23b2c797d0ee'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:31.217533 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-5cd5ef8d-22c3-47a0-b647-ca3ef8722fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74124820abce47ea93bb23b2c797d0ee'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:31.217533 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-5cd5ef8d-22c3-47a0-b647-ca3ef8722fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74124820abce47ea93bb23b2c797d0ee'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:31.217533 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-5cd5ef8d-22c3-47a0-b647-ca3ef8722fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74124820abce47ea93bb23b2c797d0ee'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:31.287440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-5cd5ef8d-22c3-47a0-b647-ca3ef8722fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74124820abce47ea93bb23b2c797d0ee'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:31.305918 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1419/2826] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:26:30 2022] DELETE /identity/v3/users/74124820abce47ea93bb23b2c797d0ee => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:31.750656 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:31.756437 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:31.767589 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaebc2-c7a3-43c9-89d6-89ad56f641c6 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:31.829762 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:31.830307 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:31.830584 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:31.830857 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] PATH_INFO: `/v3/projects/88332941eb5c4154b87069aa5aec928d` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:31.832639 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88332941eb5c4154b87069aa5aec928d)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:31.834712 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:32.058403 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88332941eb5c4154b87069aa5aec928d', 'request_id': 'req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:32.064173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88332941eb5c4154b87069aa5aec928d', 'request_id': 'req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:32.207063 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88332941eb5c4154b87069aa5aec928d', 'request_id': 'req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:32.219452 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] The token cache is being invalidate because project 88332941eb5c4154b87069aa5aec928d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:26:32.219746 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:32.220415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-bac0cda8-f3b8-4071-a4b7-923a4a8664c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:32.223399 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1408/2827] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:26:31 2022] DELETE /identity/v3/projects/88332941eb5c4154b87069aa5aec928d => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:32.228568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf0a11-2f15-43c7-b6b9-6616004f4a49 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:32.229083 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf0a11-2f15-43c7-b6b9-6616004f4a49 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:32.229473 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf0a11-2f15-43c7-b6b9-6616004f4a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:32.251527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-54bf0a11-2f15-43c7-b6b9-6616004f4a49 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:32.251756 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-54bf0a11-2f15-43c7-b6b9-6616004f4a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:32.318797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1420/2828] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:26:32.327081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc87e7-681d-4b42-ba3d-1b1a2d04401d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:32.327370 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc87e7-681d-4b42-ba3d-1b1a2d04401d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:32.327646 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc87e7-681d-4b42-ba3d-1b1a2d04401d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:32.356812 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-96dc87e7-681d-4b42-ba3d-1b1a2d04401d None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:32.358045 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-96dc87e7-681d-4b42-ba3d-1b1a2d04401d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:32.366767 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1409/2829] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:32.372047 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf0a11-2f15-43c7-b6b9-6616004f4a49 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:32.387818 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf0a11-2f15-43c7-b6b9-6616004f4a49 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:32.392841 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a01e1c-571b-4223-b2b9-922b4201f3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:32.394014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a01e1c-571b-4223-b2b9-922b4201f3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:32.394274 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a01e1c-571b-4223-b2b9-922b4201f3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:32.394534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a01e1c-571b-4223-b2b9-922b4201f3d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:32.395174 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a01e1c-571b-4223-b2b9-922b4201f3d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:26:32.395511 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a01e1c-571b-4223-b2b9-922b4201f3d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:32.397141 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a01e1c-571b-4223-b2b9-922b4201f3d3 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:32.410794 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1421/2830] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:26:32.419614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc87e7-681d-4b42-ba3d-1b1a2d04401d None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:32.429255 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc87e7-681d-4b42-ba3d-1b1a2d04401d None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:32.434487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b5b0a-1187-4e49-a745-d34ae4c10000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:32.435091 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b5b0a-1187-4e49-a745-d34ae4c10000 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:32.435374 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b5b0a-1187-4e49-a745-d34ae4c10000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:32.435645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b5b0a-1187-4e49-a745-d34ae4c10000 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:32.436294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b5b0a-1187-4e49-a745-d34ae4c10000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:32.438272 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b5b0a-1187-4e49-a745-d34ae4c10000 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:32.599494 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1410/2831] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:26:32 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:26:32.606333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a01e1c-571b-4223-b2b9-922b4201f3d3 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:32.615315 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a01e1c-571b-4223-b2b9-922b4201f3d3 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:32.620775 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:32.708969 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c2ba2-69a0-4819-aedb-2b69f95fe4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:32.710267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c2ba2-69a0-4819-aedb-2b69f95fe4dd admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:32.710267 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c2ba2-69a0-4819-aedb-2b69f95fe4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:32.710587 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c2ba2-69a0-4819-aedb-2b69f95fe4dd admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:32.711262 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c2ba2-69a0-4819-aedb-2b69f95fe4dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:32.713539 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c2ba2-69a0-4819-aedb-2b69f95fe4dd admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:32.814430 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1422/2832] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:26:32 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:26:32.819502 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b5b0a-1187-4e49-a745-d34ae4c10000 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:32.830650 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b5b0a-1187-4e49-a745-d34ae4c10000 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:32.835450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2726c6-c093-4e57-b09e-7934a95ed71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:32.836068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2726c6-c093-4e57-b09e-7934a95ed71b admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:32.836349 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2726c6-c093-4e57-b09e-7934a95ed71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:32.836623 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2726c6-c093-4e57-b09e-7934a95ed71b admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:32.837296 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2726c6-c093-4e57-b09e-7934a95ed71b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:32.838902 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2726c6-c093-4e57-b09e-7934a95ed71b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:32.849830 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1411/2833] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:26:32.854119 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c2ba2-69a0-4819-aedb-2b69f95fe4dd admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:32.866428 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c2ba2-69a0-4819-aedb-2b69f95fe4dd admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:32.871209 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ac83a-3016-49d5-b6cf-411ef13c720d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:32.871806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ac83a-3016-49d5-b6cf-411ef13c720d admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:32.872081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ac83a-3016-49d5-b6cf-411ef13c720d admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:32.872351 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ac83a-3016-49d5-b6cf-411ef13c720d admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:32.873029 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ac83a-3016-49d5-b6cf-411ef13c720d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:32.874621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ac83a-3016-49d5-b6cf-411ef13c720d admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:32.886243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1423/2834] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:26:32.890935 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2726c6-c093-4e57-b09e-7934a95ed71b admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:32.901798 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2726c6-c093-4e57-b09e-7934a95ed71b admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:32.906693 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d6b90-cb56-4031-8221-ba41f8fd4088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:32.907346 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d6b90-cb56-4031-8221-ba41f8fd4088 admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:32.907626 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d6b90-cb56-4031-8221-ba41f8fd4088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:32.907929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d6b90-cb56-4031-8221-ba41f8fd4088 admin admin] PATH_INFO: `/v3/projects/ab68ba80c53149e8b13154db35c4e2c2/users/6257debf16864da8ab702ad8092bd888/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:32.929497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d6b90-cb56-4031-8221-ba41f8fd4088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab68ba80c53149e8b13154db35c4e2c2, user_id=6257debf16864da8ab702ad8092bd888, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:32.929497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d6b90-cb56-4031-8221-ba41f8fd4088 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:33.014597 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1412/2835] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:26:32 2022] PUT /identity/v3/projects/ab68ba80c53149e8b13154db35c4e2c2/users/6257debf16864da8ab702ad8092bd888/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:33.026242 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522ee96-8c3e-4aa8-81e1-e316b256ca95 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:33.026728 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522ee96-8c3e-4aa8-81e1-e316b256ca95 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:33.027078 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522ee96-8c3e-4aa8-81e1-e316b256ca95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:33.056800 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-1522ee96-8c3e-4aa8-81e1-e316b256ca95 None None] MFA Rules not processed for user `6257debf16864da8ab702ad8092bd888`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:33.058067 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1522ee96-8c3e-4aa8-81e1-e316b256ca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:33.131444 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1424/2836] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:33.401069 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d6b90-cb56-4031-8221-ba41f8fd4088 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:33.405148 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3a7d6b90-cb56-4031-8221-ba41f8fd4088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:33.438244 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:33.466729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d6b90-cb56-4031-8221-ba41f8fd4088 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:33.471030 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e25712-12cb-43d1-8fd3-a526512cbf84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:33.472396 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e25712-12cb-43d1-8fd3-a526512cbf84 service cinder] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:33.472881 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e25712-12cb-43d1-8fd3-a526512cbf84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:33.473286 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e25712-12cb-43d1-8fd3-a526512cbf84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:33.483330 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e25712-12cb-43d1-8fd3-a526512cbf84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:33.484817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e25712-12cb-43d1-8fd3-a526512cbf84 service cinder] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:33.494406 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1413/2837] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:35.276582 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8105912-a694-46dc-85b0-2d19850f1ff3 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:35.277152 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8105912-a694-46dc-85b0-2d19850f1ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:35.277274 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8105912-a694-46dc-85b0-2d19850f1ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:35.296674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-a8105912-a694-46dc-85b0-2d19850f1ff3 None None] MFA Rules not processed for user `6257debf16864da8ab702ad8092bd888`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:26:35.297908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a8105912-a694-46dc-85b0-2d19850f1ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:35.305810 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1425/2838] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:26:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:26:43.880463 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=14 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:45.291188 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:51.243817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e25712-12cb-43d1-8fd3-a526512cbf84 service cinder] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:51.245357 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-99e25712-12cb-43d1-8fd3-a526512cbf84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:51.305258 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e25712-12cb-43d1-8fd3-a526512cbf84 service cinder] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:51.308775 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57cebb-06c6-48d1-8721-ef8bd6b28993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:51.309401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57cebb-06c6-48d1-8721-ef8bd6b28993 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:51.309698 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57cebb-06c6-48d1-8721-ef8bd6b28993 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:51.309952 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57cebb-06c6-48d1-8721-ef8bd6b28993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:51.311673 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2b57cebb-06c6-48d1-8721-ef8bd6b28993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:51.382599 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57cebb-06c6-48d1-8721-ef8bd6b28993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:51.383953 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57cebb-06c6-48d1-8721-ef8bd6b28993 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:51.394462 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1414/2839] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:53.530377 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8105912-a694-46dc-85b0-2d19850f1ff3 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:53.531925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-a8105912-a694-46dc-85b0-2d19850f1ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:53.637597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8105912-a694-46dc-85b0-2d19850f1ff3 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:53.654130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149d5e1-26c3-4078-a6ec-a1771bff3ef4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:53.654769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149d5e1-26c3-4078-a6ec-a1771bff3ef4 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:53.655102 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149d5e1-26c3-4078-a6ec-a1771bff3ef4 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:53.655395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149d5e1-26c3-4078-a6ec-a1771bff3ef4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:53.657482 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1149d5e1-26c3-4078-a6ec-a1771bff3ef4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:53.672644 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149d5e1-26c3-4078-a6ec-a1771bff3ef4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:53.674425 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149d5e1-26c3-4078-a6ec-a1771bff3ef4 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:53.688064 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1426/2840] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:54.480996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57cebb-06c6-48d1-8721-ef8bd6b28993 service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:54.483124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2b57cebb-06c6-48d1-8721-ef8bd6b28993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:54.495805 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57cebb-06c6-48d1-8721-ef8bd6b28993 service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:54.500376 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90690e6-cc74-45ef-ae37-82e54fd7bf78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:54.501083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90690e6-cc74-45ef-ae37-82e54fd7bf78 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:54.501448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90690e6-cc74-45ef-ae37-82e54fd7bf78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:54.501782 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90690e6-cc74-45ef-ae37-82e54fd7bf78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:54.504358 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-d90690e6-cc74-45ef-ae37-82e54fd7bf78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:54.519047 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90690e6-cc74-45ef-ae37-82e54fd7bf78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:54.520624 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90690e6-cc74-45ef-ae37-82e54fd7bf78 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:54.532228 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1415/2841] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:55.750370 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149d5e1-26c3-4078-a6ec-a1771bff3ef4 service nova] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:55.762016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149d5e1-26c3-4078-a6ec-a1771bff3ef4 service nova] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:55.767416 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be66f11-5e99-4cc3-816c-eb9c9512a9ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:55.768809 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66f11-5e99-4cc3-816c-eb9c9512a9ae service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:55.769322 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66f11-5e99-4cc3-816c-eb9c9512a9ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:55.769616 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66f11-5e99-4cc3-816c-eb9c9512a9ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:55.771547 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0be66f11-5e99-4cc3-816c-eb9c9512a9ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:55.787196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be66f11-5e99-4cc3-816c-eb9c9512a9ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:55.788961 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be66f11-5e99-4cc3-816c-eb9c9512a9ae service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:55.813740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1427/2842] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:27:04.527159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=56 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:05.106483 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb2e9c-baf4-42fe-b4d4-3baf6628c885 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:05.106858 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb2e9c-baf4-42fe-b4d4-3baf6628c885 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:05.107137 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb2e9c-baf4-42fe-b4d4-3baf6628c885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:05.135752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-41bb2e9c-baf4-42fe-b4d4-3baf6628c885 None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:27:05.137026 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-41bb2e9c-baf4-42fe-b4d4-3baf6628c885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:05.196276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1416/2843] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:27:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:27:05.218411 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be66f11-5e99-4cc3-816c-eb9c9512a9ae service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:05.229780 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0be66f11-5e99-4cc3-816c-eb9c9512a9ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:05.294081 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be66f11-5e99-4cc3-816c-eb9c9512a9ae service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:05.300475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120638ce-2383-4a40-8ac4-0497ec9ebf5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:05.301714 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120638ce-2383-4a40-8ac4-0497ec9ebf5a service swift] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:05.302004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120638ce-2383-4a40-8ac4-0497ec9ebf5a service swift] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:05.302279 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120638ce-2383-4a40-8ac4-0497ec9ebf5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:05.312257 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120638ce-2383-4a40-8ac4-0497ec9ebf5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:05.313873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120638ce-2383-4a40-8ac4-0497ec9ebf5a service swift] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:05.322635 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1428/2844] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:27:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 10:27:06.263783 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b4c06-0d65-4fda-8ac7-a0cdd09b6a6f None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:06.264345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b4c06-0d65-4fda-8ac7-a0cdd09b6a6f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:06.264345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b4c06-0d65-4fda-8ac7-a0cdd09b6a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:06.287858 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-a89b4c06-0d65-4fda-8ac7-a0cdd09b6a6f None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:27:06.289201 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a89b4c06-0d65-4fda-8ac7-a0cdd09b6a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:06.312267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1417/2845] 10.222.0.30 () {58 vars in 967 bytes} [Wed Aug 24 10:27:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:27:06.313182 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120638ce-2383-4a40-8ac4-0497ec9ebf5a service swift] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:06.330633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120638ce-2383-4a40-8ac4-0497ec9ebf5a service swift] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:06.340906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fad59a-10e1-441f-bf2c-0cbee919b95f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:06.341551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fad59a-10e1-441f-bf2c-0cbee919b95f service swift] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:06.341854 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fad59a-10e1-441f-bf2c-0cbee919b95f service swift] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:06.342130 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fad59a-10e1-441f-bf2c-0cbee919b95f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:06.356704 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fad59a-10e1-441f-bf2c-0cbee919b95f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:06.358600 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fad59a-10e1-441f-bf2c-0cbee919b95f service swift] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:06.369627 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1429/2846] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:27:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 10:27:07.248707 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdc985-19c6-4492-a381-0318bf3b5d8d None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:07.249212 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdc985-19c6-4492-a381-0318bf3b5d8d None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:07.249363 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdc985-19c6-4492-a381-0318bf3b5d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:07.275475 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-66cdc985-19c6-4492-a381-0318bf3b5d8d None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:27:07.276897 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-66cdc985-19c6-4492-a381-0318bf3b5d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:07.284182 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1418/2847] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Aug 24 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:27:07.296825 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fad59a-10e1-441f-bf2c-0cbee919b95f service swift] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:07.307577 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fad59a-10e1-441f-bf2c-0cbee919b95f service swift] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:07.312943 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bd39-45a9-4ca9-a494-122cefde803d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:07.313543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bd39-45a9-4ca9-a494-122cefde803d service swift] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:07.314082 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bd39-45a9-4ca9-a494-122cefde803d service swift] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:07.315435 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bd39-45a9-4ca9-a494-122cefde803d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:07.323024 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97bd39-45a9-4ca9-a494-122cefde803d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:07.324708 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97bd39-45a9-4ca9-a494-122cefde803d service swift] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:07.335474 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1430/2848] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:27:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:27:08.043645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5272f-876b-452f-b662-d4ebbccd8e92 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:08.043645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5272f-876b-452f-b662-d4ebbccd8e92 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:08.043645 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5272f-876b-452f-b662-d4ebbccd8e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:08.056004 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-fad5272f-876b-452f-b662-d4ebbccd8e92 None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:27:08.057285 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-fad5272f-876b-452f-b662-d4ebbccd8e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:08.064368 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1419/2849] 10.222.0.30 () {58 vars in 967 bytes} [Wed Aug 24 10:27:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:27:08.076975 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bd39-45a9-4ca9-a494-122cefde803d service swift] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:08.088183 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bd39-45a9-4ca9-a494-122cefde803d service swift] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:08.093307 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8ca0e-c3a1-4487-bb92-26f58a69fb81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:08.095410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8ca0e-c3a1-4487-bb92-26f58a69fb81 service swift] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:08.095410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8ca0e-c3a1-4487-bb92-26f58a69fb81 service swift] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:08.095410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8ca0e-c3a1-4487-bb92-26f58a69fb81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:08.102947 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8ca0e-c3a1-4487-bb92-26f58a69fb81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:08.104556 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8ca0e-c3a1-4487-bb92-26f58a69fb81 service swift] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:08.113569 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1431/2850] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:27:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:27:12.038235 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5272f-876b-452f-b662-d4ebbccd8e92 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:12.108565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5272f-876b-452f-b662-d4ebbccd8e92 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:12.114146 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:12.115275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:12.115275 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:12.115503 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] PATH_INFO: `/v3/users/6257debf16864da8ab702ad8092bd888` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:12.116629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6257debf16864da8ab702ad8092bd888)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:12.118591 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:12.399289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6257debf16864da8ab702ad8092bd888'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:27:12.399289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6257debf16864da8ab702ad8092bd888'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:27:12.403733 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6257debf16864da8ab702ad8092bd888'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:27:12.420040 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6257debf16864da8ab702ad8092bd888'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:27:12.572594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6257debf16864da8ab702ad8092bd888'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:27:12.578929 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1420/2851] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:27:12 2022] DELETE /identity/v3/users/6257debf16864da8ab702ad8092bd888 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:27:12.584543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c8c59-520b-4db3-9db7-6bb6412b694b None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:12.585095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c8c59-520b-4db3-9db7-6bb6412b694b None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:12.585695 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c8c59-520b-4db3-9db7-6bb6412b694b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:12.596281 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=100 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:12.607796 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-502c8c59-520b-4db3-9db7-6bb6412b694b None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:27:12.608842 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-502c8c59-520b-4db3-9db7-6bb6412b694b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:12.675873 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1432/2852] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:27:12.686691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:12.688178 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:12.695958 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:12.751443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915d96e-0f2f-4d4e-89bd-cd7b16954136 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:12.755289 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910cd67-4793-4dc0-8de1-cad73a38c690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:12.755922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910cd67-4793-4dc0-8de1-cad73a38c690 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:12.756206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910cd67-4793-4dc0-8de1-cad73a38c690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:12.756497 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910cd67-4793-4dc0-8de1-cad73a38c690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:12.766818 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910cd67-4793-4dc0-8de1-cad73a38c690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:12.770873 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910cd67-4793-4dc0-8de1-cad73a38c690 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:12.778556 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1421/2853] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:27:13.049333 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c8c59-520b-4db3-9db7-6bb6412b694b None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:13.051001 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-502c8c59-520b-4db3-9db7-6bb6412b694b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:13.065147 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c8c59-520b-4db3-9db7-6bb6412b694b None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:13.070601 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:13.071361 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:13.071640 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:13.071909 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] PATH_INFO: `/v3/projects/ab68ba80c53149e8b13154db35c4e2c2` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:13.073218 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab68ba80c53149e8b13154db35c4e2c2)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:13.075226 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:13.210752 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab68ba80c53149e8b13154db35c4e2c2', 'request_id': 'req-b04d502d-ff77-4804-a984-2e31dab321a8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:27:13.216466 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab68ba80c53149e8b13154db35c4e2c2', 'request_id': 'req-b04d502d-ff77-4804-a984-2e31dab321a8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:27:13.222776 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab68ba80c53149e8b13154db35c4e2c2', 'request_id': 'req-b04d502d-ff77-4804-a984-2e31dab321a8'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:27:13.280475 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] The token cache is being invalidate because project ab68ba80c53149e8b13154db35c4e2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:27:13.280762 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:27:13.281412 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:27:13.282880 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1433/2854] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:27:13 2022] DELETE /identity/v3/projects/ab68ba80c53149e8b13154db35c4e2c2 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:27:18.227146 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafe3f1-3b36-4552-b07d-c38319101d6f None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:18.227653 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafe3f1-3b36-4552-b07d-c38319101d6f None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:18.227753 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafe3f1-3b36-4552-b07d-c38319101d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:18.246464 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-4aafe3f1-3b36-4552-b07d-c38319101d6f None None] MFA Rules not processed for user `82c3bba1fcac455684cc0c8190f8231e`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:27:18.247666 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4aafe3f1-3b36-4552-b07d-c38319101d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:18.303554 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1422/2855] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:27:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:27:20.985693 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:20.988797 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:21.049926 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d502d-ff77-4804-a984-2e31dab321a8 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:21.054184 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97afe24-46b1-4797-8f4b-2a8d3d75a21a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:21.054792 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97afe24-46b1-4797-8f4b-2a8d3d75a21a service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:21.055074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97afe24-46b1-4797-8f4b-2a8d3d75a21a service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:21.055346 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97afe24-46b1-4797-8f4b-2a8d3d75a21a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:21.057142 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-c97afe24-46b1-4797-8f4b-2a8d3d75a21a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:21.068140 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97afe24-46b1-4797-8f4b-2a8d3d75a21a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:21.069616 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97afe24-46b1-4797-8f4b-2a8d3d75a21a service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:21.078548 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1434/2856] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:27:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:27:28.296810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=49 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:31.074342 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:59.873489 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafe3f1-3b36-4552-b07d-c38319101d6f None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:59.874892 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-4aafe3f1-3b36-4552-b07d-c38319101d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:59.950454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafe3f1-3b36-4552-b07d-c38319101d6f None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:59.955967 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78d7ad-4146-4c91-8913-439faa89a492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:59.956574 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78d7ad-4146-4c91-8913-439faa89a492 service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:59.956892 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78d7ad-4146-4c91-8913-439faa89a492 service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:59.957490 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78d7ad-4146-4c91-8913-439faa89a492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:59.958951 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7a78d7ad-4146-4c91-8913-439faa89a492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:27:59.971366 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78d7ad-4146-4c91-8913-439faa89a492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:59.973714 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78d7ad-4146-4c91-8913-439faa89a492 service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:59.985487 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1423/2857] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:27:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:28:09.981220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=34 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:26.620596 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8170f7-0b2d-4497-8341-ef53beba4be6 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:26.621115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8170f7-0b2d-4497-8341-ef53beba4be6 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:26.621339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8170f7-0b2d-4497-8341-ef53beba4be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:26.670295 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-2a8170f7-0b2d-4497-8341-ef53beba4be6 None None] MFA Rules not processed for user `8b7227998e7846c7a9bc14b12b637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:28:26.671507 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2a8170f7-0b2d-4497-8341-ef53beba4be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:26.738679 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1435/2858] 10.222.0.30 () {58 vars in 967 bytes} [Wed Aug 24 10:28:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:28:26.940576 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78d7ad-4146-4c91-8913-439faa89a492 service nova] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:26.942369 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-7a78d7ad-4146-4c91-8913-439faa89a492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:27.035220 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78d7ad-4146-4c91-8913-439faa89a492 service nova] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:27.041229 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff02b9f-b3a8-4feb-b779-42427c71ece8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fe81f7b4c04e9f9a26b430b39ccd36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:27.041827 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff02b9f-b3a8-4feb-b779-42427c71ece8 service swift] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:27.042104 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff02b9f-b3a8-4feb-b779-42427c71ece8 service swift] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:27.042373 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff02b9f-b3a8-4feb-b779-42427c71ece8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:27.050638 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff02b9f-b3a8-4feb-b779-42427c71ece8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:27.052547 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff02b9f-b3a8-4feb-b779-42427c71ece8 service swift] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:27.060569 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1424/2859] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Aug 24 10:28:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:28:29.311281 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8170f7-0b2d-4497-8341-ef53beba4be6 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:29.312822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2a8170f7-0b2d-4497-8341-ef53beba4be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:29.382879 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8170f7-0b2d-4497-8341-ef53beba4be6 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:29.387453 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:29.388086 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:29.388367 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:29.388642 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] PATH_INFO: `/v3/users/82c3bba1fcac455684cc0c8190f8231e` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:29.389822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c3bba1fcac455684cc0c8190f8231e)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:29.391737 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:29.587970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c3bba1fcac455684cc0c8190f8231e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:29.588305 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c3bba1fcac455684cc0c8190f8231e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:29.596593 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c3bba1fcac455684cc0c8190f8231e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:29.598112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c3bba1fcac455684cc0c8190f8231e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:29.665210 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c3bba1fcac455684cc0c8190f8231e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:29.675559 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:29.690026 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1436/2860] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:28:29 2022] DELETE /identity/v3/users/82c3bba1fcac455684cc0c8190f8231e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:28:29.957647 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff02b9f-b3a8-4feb-b779-42427c71ece8 service swift] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:29.977005 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff02b9f-b3a8-4feb-b779-42427c71ece8 service swift] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:30.043737 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:30.044756 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:30.045254 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:30.045718 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] PATH_INFO: `/v3/projects/7d5efdccb2b64693a0a950c711114f3a` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:30.046825 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5efdccb2b64693a0a950c711114f3a)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:30.048893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:30.161387 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5efdccb2b64693a0a950c711114f3a', 'request_id': 'req-45affe58-6910-4dfa-b501-fefc804af041'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:30.172963 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5efdccb2b64693a0a950c711114f3a', 'request_id': 'req-45affe58-6910-4dfa-b501-fefc804af041'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:30.281933 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5efdccb2b64693a0a950c711114f3a', 'request_id': 'req-45affe58-6910-4dfa-b501-fefc804af041'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:30.300101 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:30.305411 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] The token cache is being invalidate because project 7d5efdccb2b64693a0a950c711114f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:28:30.305706 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:30.306382 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:30.307932 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1425/2861] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:28:29 2022] DELETE /identity/v3/projects/7d5efdccb2b64693a0a950c711114f3a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:28:35.584471 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:35.585971 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:35.600072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f71a5-b160-4b4c-abfb-a81a0e244d94 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:35.656447 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:35.657103 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:35.657385 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:35.657659 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] PATH_INFO: `/v3/users/8744189102a44d47a3b994bfe49fd8e1` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:35.658773 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8744189102a44d47a3b994bfe49fd8e1)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:35.660668 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:35.911023 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8744189102a44d47a3b994bfe49fd8e1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:35.911599 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8744189102a44d47a3b994bfe49fd8e1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:35.919190 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8744189102a44d47a3b994bfe49fd8e1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:35.919822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8744189102a44d47a3b994bfe49fd8e1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.004869 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8744189102a44d47a3b994bfe49fd8e1'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.022786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1437/2862] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:28:35 2022] DELETE /identity/v3/users/8744189102a44d47a3b994bfe49fd8e1 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:28:36.269690 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:36.271145 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:36.286104 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45affe58-6910-4dfa-b501-fefc804af041 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:36.345314 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:36.345943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:36.346221 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:36.346495 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] PATH_INFO: `/v3/projects/e9980d2c52934ca38dfb4d3dfbf9700b` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:36.347614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9980d2c52934ca38dfb4d3dfbf9700b)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:36.349548 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:36.462994 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9980d2c52934ca38dfb4d3dfbf9700b', 'request_id': 'req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.468524 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9980d2c52934ca38dfb4d3dfbf9700b', 'request_id': 'req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.547692 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9980d2c52934ca38dfb4d3dfbf9700b', 'request_id': 'req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.560501 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] The token cache is being invalidate because project e9980d2c52934ca38dfb4d3dfbf9700b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:28:36.560795 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.561465 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.562943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1426/2863] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:28:36 2022] DELETE /identity/v3/projects/e9980d2c52934ca38dfb4d3dfbf9700b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:28:36.567051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:36.568568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:36.583074 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f0db8-38d8-47d9-a988-f5ac391dd6cb admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:36.641410 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0811f-a2a8-483f-8210-5cc0d1e874be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:36.642060 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0811f-a2a8-483f-8210-5cc0d1e874be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:36.642339 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0811f-a2a8-483f-8210-5cc0d1e874be admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:36.642614 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0811f-a2a8-483f-8210-5cc0d1e874be admin admin] PATH_INFO: `/v3/users/c79287b96d904360ba51048977e2166e` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:36.643780 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0811f-a2a8-483f-8210-5cc0d1e874be admin admin] RBAC: Authorizing `identity:delete_user(user_id=c79287b96d904360ba51048977e2166e)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:36.645758 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0811f-a2a8-483f-8210-5cc0d1e874be admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:36.658674 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:36.825894 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-c6c0811f-a2a8-483f-8210-5cc0d1e874be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c79287b96d904360ba51048977e2166e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.826477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-c6c0811f-a2a8-483f-8210-5cc0d1e874be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c79287b96d904360ba51048977e2166e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.833292 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-c6c0811f-a2a8-483f-8210-5cc0d1e874be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c79287b96d904360ba51048977e2166e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.833916 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-c6c0811f-a2a8-483f-8210-5cc0d1e874be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c79287b96d904360ba51048977e2166e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.892707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-c6c0811f-a2a8-483f-8210-5cc0d1e874be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c79287b96d904360ba51048977e2166e'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:36.911988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1438/2864] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:28:36 2022] DELETE /identity/v3/users/c79287b96d904360ba51048977e2166e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:28:37.162763 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:37.164315 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:37.177326 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e6c9f-3bb6-4b12-b8bd-145a927d5e03 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:37.237918 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:37.238541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:37.238817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:37.239083 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] PATH_INFO: `/v3/projects/a1d84b385b5e41d38f686196688fb8ba` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:37.240170 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1d84b385b5e41d38f686196688fb8ba)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:37.242282 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:37.350249 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:37.362726 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1d84b385b5e41d38f686196688fb8ba', 'request_id': 'req-76d88759-4634-4439-bb19-7fc96bbdbbe0'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:37.368276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1d84b385b5e41d38f686196688fb8ba', 'request_id': 'req-76d88759-4634-4439-bb19-7fc96bbdbbe0'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:37.469943 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1d84b385b5e41d38f686196688fb8ba', 'request_id': 'req-76d88759-4634-4439-bb19-7fc96bbdbbe0'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:37.482154 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] The token cache is being invalidate because project a1d84b385b5e41d38f686196688fb8ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81930) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:28:37.482810 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:37.483544 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-76d88759-4634-4439-bb19-7fc96bbdbbe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:28:37.485056 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1427/2865] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:28:37 2022] DELETE /identity/v3/projects/a1d84b385b5e41d38f686196688fb8ba => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:28:37.490009 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbd5d3-9152-4686-9d26-9f849320ce52 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:37.490327 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbd5d3-9152-4686-9d26-9f849320ce52 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:37.490606 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbd5d3-9152-4686-9d26-9f849320ce52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:37.510268 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-9ecbd5d3-9152-4686-9d26-9f849320ce52 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:28:37.511476 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-9ecbd5d3-9152-4686-9d26-9f849320ce52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:37.571113 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1439/2866] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:28:37.579567 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071b962-bf6d-4738-9883-6cccbc024245 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:37.579853 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071b962-bf6d-4738-9883-6cccbc024245 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:37.580123 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071b962-bf6d-4738-9883-6cccbc024245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:37.600081 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-0071b962-bf6d-4738-9883-6cccbc024245 None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:28:37.601345 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-0071b962-bf6d-4738-9883-6cccbc024245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:37.609615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1428/2867] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:28:37.615933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbd5d3-9152-4686-9d26-9f849320ce52 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:37.625807 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbd5d3-9152-4686-9d26-9f849320ce52 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:37.630434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ec553-8e42-4a56-ab7d-1328a49d40bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:37.631059 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec553-8e42-4a56-ab7d-1328a49d40bc admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:37.631336 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec553-8e42-4a56-ab7d-1328a49d40bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:37.631607 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec553-8e42-4a56-ab7d-1328a49d40bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:37.632246 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ec553-8e42-4a56-ab7d-1328a49d40bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81931) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:28:37.632552 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ec553-8e42-4a56-ab7d-1328a49d40bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:37.634786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ec553-8e42-4a56-ab7d-1328a49d40bc admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:37.646260 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1440/2868] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Aug 24 10:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:28:37.651334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071b962-bf6d-4738-9883-6cccbc024245 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:37.660541 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071b962-bf6d-4738-9883-6cccbc024245 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:37.665602 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69707ec4-7ee0-4bc2-a19e-c2ab294c79f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:37.666198 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69707ec4-7ee0-4bc2-a19e-c2ab294c79f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:37.666478 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69707ec4-7ee0-4bc2-a19e-c2ab294c79f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:37.666749 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69707ec4-7ee0-4bc2-a19e-c2ab294c79f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:37.667375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69707ec4-7ee0-4bc2-a19e-c2ab294c79f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:37.669331 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69707ec4-7ee0-4bc2-a19e-c2ab294c79f5 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:37.763283 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1429/2869] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:28:37 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:28:37.767520 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ec553-8e42-4a56-ab7d-1328a49d40bc admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:37.778815 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ec553-8e42-4a56-ab7d-1328a49d40bc admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:37.836036 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a5614-606d-4639-b007-505ea293bd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:37.836806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a5614-606d-4639-b007-505ea293bd6e admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:37.837153 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a5614-606d-4639-b007-505ea293bd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:37.837434 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a5614-606d-4639-b007-505ea293bd6e admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:37.838095 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a5614-606d-4639-b007-505ea293bd6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:37.840001 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a5614-606d-4639-b007-505ea293bd6e admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:37.921786 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1441/2870] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:28:37 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:28:37.925895 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69707ec4-7ee0-4bc2-a19e-c2ab294c79f5 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:37.937415 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69707ec4-7ee0-4bc2-a19e-c2ab294c79f5 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:37.942003 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb1158-2679-48dd-88dc-10b45fca6d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:37.942594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb1158-2679-48dd-88dc-10b45fca6d1c admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:37.942875 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb1158-2679-48dd-88dc-10b45fca6d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:37.943146 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb1158-2679-48dd-88dc-10b45fca6d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:37.943780 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb1158-2679-48dd-88dc-10b45fca6d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:37.945450 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb1158-2679-48dd-88dc-10b45fca6d1c admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:37.956261 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1430/2871] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:28:37.960384 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a5614-606d-4639-b007-505ea293bd6e admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:37.970477 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a5614-606d-4639-b007-505ea293bd6e admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:37.975076 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602ebbc-d0b6-40b8-9c1b-16d7eb45b08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:37.975662 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602ebbc-d0b6-40b8-9c1b-16d7eb45b08b admin admin] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:37.975938 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602ebbc-d0b6-40b8-9c1b-16d7eb45b08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:37.976206 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602ebbc-d0b6-40b8-9c1b-16d7eb45b08b admin admin] PATH_INFO: `/v3/roles` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:37.976922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602ebbc-d0b6-40b8-9c1b-16d7eb45b08b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:37.978763 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602ebbc-d0b6-40b8-9c1b-16d7eb45b08b admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:37.980115 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:38.001515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1442/2872] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:28:38.001759 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb1158-2679-48dd-88dc-10b45fca6d1c admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:38.006634 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb1158-2679-48dd-88dc-10b45fca6d1c admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:38.012023 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543750f5-a68d-4a64-b93a-709004218f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:38.013037 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543750f5-a68d-4a64-b93a-709004218f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:38.013328 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543750f5-a68d-4a64-b93a-709004218f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:38.013607 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543750f5-a68d-4a64-b93a-709004218f2b admin admin] PATH_INFO: `/v3/projects/8013ed9ae5ed46b9bc656f193c43e118/users/a71f15d0547a4d78b6d540e5eb5b2fbc/roles/13913230583d4ce384599ac5dace7ffa` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:38.031187 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543750f5-a68d-4a64-b93a-709004218f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8013ed9ae5ed46b9bc656f193c43e118, user_id=a71f15d0547a4d78b6d540e5eb5b2fbc, role_id=13913230583d4ce384599ac5dace7ffa)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:38.033878 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543750f5-a68d-4a64-b93a-709004218f2b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:38.120149 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1431/2873] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:28:37 2022] PUT /identity/v3/projects/8013ed9ae5ed46b9bc656f193c43e118/users/a71f15d0547a4d78b6d540e5eb5b2fbc/roles/13913230583d4ce384599ac5dace7ffa => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:28:38.125534 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db248e6-2f0d-4e0e-92da-c341aa8b1f5c None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:38.125859 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db248e6-2f0d-4e0e-92da-c341aa8b1f5c None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:38.126137 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db248e6-2f0d-4e0e-92da-c341aa8b1f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:38.145254 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-0db248e6-2f0d-4e0e-92da-c341aa8b1f5c None None] MFA Rules not processed for user `deb4aa9488e54d0eafc940ddd57f345d`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:28:38.146458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0db248e6-2f0d-4e0e-92da-c341aa8b1f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:38.204738 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1443/2874] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:28:38.214847 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543750f5-a68d-4a64-b93a-709004218f2b admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:38.216616 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-543750f5-a68d-4a64-b93a-709004218f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:38.274611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543750f5-a68d-4a64-b93a-709004218f2b admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:38.278276 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce39c41-8c9e-4ab8-bca4-b155198efc7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:38.278791 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce39c41-8c9e-4ab8-bca4-b155198efc7f service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:38.279067 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce39c41-8c9e-4ab8-bca4-b155198efc7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:38.279334 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce39c41-8c9e-4ab8-bca4-b155198efc7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:38.288206 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce39c41-8c9e-4ab8-bca4-b155198efc7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:38.293910 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce39c41-8c9e-4ab8-bca4-b155198efc7f service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:38.301504 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1432/2875] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:28:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:28:43.684404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73168a66-fa77-4216-bf8c-09570ebe7158 None None] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:43.684966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73168a66-fa77-4216-bf8c-09570ebe7158 None None] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:43.684966 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73168a66-fa77-4216-bf8c-09570ebe7158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:43.723516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.auth.core [None req-73168a66-fa77-4216-bf8c-09570ebe7158 None None] MFA Rules not processed for user `a71f15d0547a4d78b6d540e5eb5b2fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81931) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:28:43.725051 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-73168a66-fa77-4216-bf8c-09570ebe7158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:43.807981 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1444/2876] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:28:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:28:43.814401 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce39c41-8c9e-4ab8-bca4-b155198efc7f service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:43.832016 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce39c41-8c9e-4ab8-bca4-b155198efc7f service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:43.837800 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307cd1e-1c95-4831-9e7d-37cc3a415dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:43.838426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307cd1e-1c95-4831-9e7d-37cc3a415dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:43.838713 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307cd1e-1c95-4831-9e7d-37cc3a415dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:43.838996 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307cd1e-1c95-4831-9e7d-37cc3a415dd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:43.839776 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307cd1e-1c95-4831-9e7d-37cc3a415dd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:43.842586 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307cd1e-1c95-4831-9e7d-37cc3a415dd6 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:43.975013 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1433/2877] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Aug 24 10:28:43 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:28:43.979621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73168a66-fa77-4216-bf8c-09570ebe7158 None None] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:43.990748 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73168a66-fa77-4216-bf8c-09570ebe7158 None None] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:44.047845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca473f-566f-40be-9836-8ebdaad42bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:44.048414 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca473f-566f-40be-9836-8ebdaad42bfb admin admin] REQUEST_METHOD: `POST` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:44.048697 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca473f-566f-40be-9836-8ebdaad42bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:44.049008 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca473f-566f-40be-9836-8ebdaad42bfb admin admin] PATH_INFO: `/v3/users` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:44.049660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca473f-566f-40be-9836-8ebdaad42bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:44.051557 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca473f-566f-40be-9836-8ebdaad42bfb admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:44.125845 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1445/2878] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Aug 24 10:28:43 2022] POST /identity/v3/users => generated 296 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:28:44.129789 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307cd1e-1c95-4831-9e7d-37cc3a415dd6 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:44.141119 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307cd1e-1c95-4831-9e7d-37cc3a415dd6 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:44.145617 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3d552-8df5-4eea-9f23-65da07143b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:44.146203 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3d552-8df5-4eea-9f23-65da07143b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:44.146479 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3d552-8df5-4eea-9f23-65da07143b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:44.146752 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3d552-8df5-4eea-9f23-65da07143b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:44.147376 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3d552-8df5-4eea-9f23-65da07143b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:44.149133 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3d552-8df5-4eea-9f23-65da07143b05 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:44.160565 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1434/2879] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:28:44.165543 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca473f-566f-40be-9836-8ebdaad42bfb admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:44.175717 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca473f-566f-40be-9836-8ebdaad42bfb admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:44.181347 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b001ae-621b-4f79-90f5-a91ae5b458c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:44.181980 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b001ae-621b-4f79-90f5-a91ae5b458c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:44.182259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b001ae-621b-4f79-90f5-a91ae5b458c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:44.182551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b001ae-621b-4f79-90f5-a91ae5b458c0 admin admin] PATH_INFO: `/v3/projects/64b205bd5ef941a4a38182871d4cdbde/users/b6639eb8aa1b4b7db0e559a9d155c9a3/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:44.194404 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:44.197862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b001ae-621b-4f79-90f5-a91ae5b458c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b205bd5ef941a4a38182871d4cdbde, user_id=b6639eb8aa1b4b7db0e559a9d155c9a3, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:44.200741 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b001ae-621b-4f79-90f5-a91ae5b458c0 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:44.288162 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1446/2880] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:28:44 2022] PUT /identity/v3/projects/64b205bd5ef941a4a38182871d4cdbde/users/b6639eb8aa1b4b7db0e559a9d155c9a3/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:28:44.293219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3d552-8df5-4eea-9f23-65da07143b05 admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:44.306729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3d552-8df5-4eea-9f23-65da07143b05 admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:44.321180 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:44.364815 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc9efa-d559-47ff-9d82-0e14932f7710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:44.365492 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc9efa-d559-47ff-9d82-0e14932f7710 admin admin] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:44.365786 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc9efa-d559-47ff-9d82-0e14932f7710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:44.366060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc9efa-d559-47ff-9d82-0e14932f7710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:44.366691 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc9efa-d559-47ff-9d82-0e14932f7710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:44.368511 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc9efa-d559-47ff-9d82-0e14932f7710 admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:44.378728 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1435/2881] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Aug 24 10:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:28:44.383527 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b001ae-621b-4f79-90f5-a91ae5b458c0 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:44.394518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b001ae-621b-4f79-90f5-a91ae5b458c0 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:44.400546 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f86258-ce02-4710-86fe-5291cfc18513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:44.401230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86258-ce02-4710-86fe-5291cfc18513 admin admin] REQUEST_METHOD: `PUT` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:44.401509 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86258-ce02-4710-86fe-5291cfc18513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:44.401782 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86258-ce02-4710-86fe-5291cfc18513 admin admin] PATH_INFO: `/v3/projects/64b205bd5ef941a4a38182871d4cdbde/users/b6639eb8aa1b4b7db0e559a9d155c9a3/roles/2a679dbfb6674ce88a681a353738cb1f` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:44.403744 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f86258-ce02-4710-86fe-5291cfc18513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b205bd5ef941a4a38182871d4cdbde, user_id=b6639eb8aa1b4b7db0e559a9d155c9a3, role_id=2a679dbfb6674ce88a681a353738cb1f)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:44.406506 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f86258-ce02-4710-86fe-5291cfc18513 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:44.417785 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1447/2882] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Aug 24 10:28:44 2022] PUT /identity/v3/projects/64b205bd5ef941a4a38182871d4cdbde/users/b6639eb8aa1b4b7db0e559a9d155c9a3/roles/2a679dbfb6674ce88a681a353738cb1f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:28:50.171060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce1827-b67b-4a67-a853-cf3d426716c5 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:50.171556 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce1827-b67b-4a67-a853-cf3d426716c5 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:50.171667 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce1827-b67b-4a67-a853-cf3d426716c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:50.201158 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-16ce1827-b67b-4a67-a853-cf3d426716c5 None None] MFA Rules not processed for user `b6639eb8aa1b4b7db0e559a9d155c9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:28:50.202356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-16ce1827-b67b-4a67-a853-cf3d426716c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:50.274736 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1436/2883] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:28:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:28:50.291570 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f86258-ce02-4710-86fe-5291cfc18513 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:50.293331 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-77f86258-ce02-4710-86fe-5291cfc18513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:28:50.366192 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f86258-ce02-4710-86fe-5291cfc18513 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:50.372324 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03371b73-6cea-4e47-9a80-0e9b8fbb1313 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:50.372965 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03371b73-6cea-4e47-9a80-0e9b8fbb1313 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:50.378004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03371b73-6cea-4e47-9a80-0e9b8fbb1313 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:50.378004 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03371b73-6cea-4e47-9a80-0e9b8fbb1313 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:50.428917 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03371b73-6cea-4e47-9a80-0e9b8fbb1313 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:50.431225 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03371b73-6cea-4e47-9a80-0e9b8fbb1313 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:50.440821 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1448/2884] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:28:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:29:00.271433 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=55 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:00.436464 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=64,keystone:INSERT=2 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:09.122799 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ef425-0ebe-4fd7-bfd6-8594f25c2ea6 None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:29:09.123331 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ef425-0ebe-4fd7-bfd6-8594f25c2ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:29:09.123772 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ef425-0ebe-4fd7-bfd6-8594f25c2ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:29:09.146530 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-5b3ef425-0ebe-4fd7-bfd6-8594f25c2ea6 None None] MFA Rules not processed for user `a71f15d0547a4d78b6d540e5eb5b2fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:29:09.147732 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5b3ef425-0ebe-4fd7-bfd6-8594f25c2ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:29:09.154971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1437/2885] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:29:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:29:09.495171 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03371b73-6cea-4e47-9a80-0e9b8fbb1313 service nova] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:29:09.552424 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03371b73-6cea-4e47-9a80-0e9b8fbb1313 service nova] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:29:09.556105 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07e12e-39d7-49f1-a520-1f386d44630b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:29:09.556707 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07e12e-39d7-49f1-a520-1f386d44630b service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:29:09.557019 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07e12e-39d7-49f1-a520-1f386d44630b service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:29:09.557290 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07e12e-39d7-49f1-a520-1f386d44630b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:29:09.559760 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-db07e12e-39d7-49f1-a520-1f386d44630b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:29:09.569895 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07e12e-39d7-49f1-a520-1f386d44630b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:29:09.571308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07e12e-39d7-49f1-a520-1f386d44630b service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:29:09.580163 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1449/2886] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:29:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:29:10.479537 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ef425-0ebe-4fd7-bfd6-8594f25c2ea6 None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:29:10.481393 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5b3ef425-0ebe-4fd7-bfd6-8594f25c2ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:29:10.559174 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ef425-0ebe-4fd7-bfd6-8594f25c2ea6 None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:29:10.564191 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79360e-077c-4ad2-94df-9edc52a889ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:29:10.564793 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79360e-077c-4ad2-94df-9edc52a889ea service placement] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:29:10.565102 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79360e-077c-4ad2-94df-9edc52a889ea service placement] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:29:10.565375 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79360e-077c-4ad2-94df-9edc52a889ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:29:10.567161 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5a79360e-077c-4ad2-94df-9edc52a889ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:29:10.579971 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79360e-077c-4ad2-94df-9edc52a889ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:29:10.581614 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79360e-077c-4ad2-94df-9edc52a889ea service placement] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:29:10.591972 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1438/2887] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:29:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:29:19.576079 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=26 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:20.587611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=35 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:25.278014 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07e12e-39d7-49f1-a520-1f386d44630b service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:29:25.291822 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07e12e-39d7-49f1-a520-1f386d44630b service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:29:25.298710 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7066d-085f-4bec-9b68-1833b7bb24ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:29:25.299285 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7066d-085f-4bec-9b68-1833b7bb24ab service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:29:25.299769 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7066d-085f-4bec-9b68-1833b7bb24ab service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:29:25.300230 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7066d-085f-4bec-9b68-1833b7bb24ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:29:25.302978 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5ba7066d-085f-4bec-9b68-1833b7bb24ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:29:25.318597 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7066d-085f-4bec-9b68-1833b7bb24ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:29:25.320191 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7066d-085f-4bec-9b68-1833b7bb24ab service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:29:25.334313 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1450/2888] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:29:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:29:28.210434 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79360e-077c-4ad2-94df-9edc52a889ea service placement] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:29:28.211906 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-5a79360e-077c-4ad2-94df-9edc52a889ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:29:28.224677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79360e-077c-4ad2-94df-9edc52a889ea service placement] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:29:28.228426 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d022d8c-1a71-4a10-b2d2-ccbae631d007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:29:28.229059 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022d8c-1a71-4a10-b2d2-ccbae631d007 service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:29:28.229342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022d8c-1a71-4a10-b2d2-ccbae631d007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:29:28.229817 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022d8c-1a71-4a10-b2d2-ccbae631d007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:29:28.239294 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d022d8c-1a71-4a10-b2d2-ccbae631d007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:29:28.240723 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d022d8c-1a71-4a10-b2d2-ccbae631d007 service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:29:28.250367 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1439/2889] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:29:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:29:35.328198 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=6 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:38.246543 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=6 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:10.998621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7066d-085f-4bec-9b68-1833b7bb24ab service nova] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:31:11.000018 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-5ba7066d-085f-4bec-9b68-1833b7bb24ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:31:11.014621 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7066d-085f-4bec-9b68-1833b7bb24ab service nova] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:31:11.020723 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2833b2-eb12-494b-bcc5-213c43421da4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:31:11.021360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2833b2-eb12-494b-bcc5-213c43421da4 service placement] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:31:11.021639 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2833b2-eb12-494b-bcc5-213c43421da4 service placement] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:31:11.021910 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2833b2-eb12-494b-bcc5-213c43421da4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:31:11.023629 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-0c2833b2-eb12-494b-bcc5-213c43421da4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:31:11.036016 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2833b2-eb12-494b-bcc5-213c43421da4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:31:11.037608 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2833b2-eb12-494b-bcc5-213c43421da4 service placement] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:31:11.047395 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1451/2890] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:31:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:31:21.043133 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=6 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:30.578159 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57937a-230a-4ec1-b6a6-0b312a8317cb None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:31:30.578677 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57937a-230a-4ec1-b6a6-0b312a8317cb None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:31:30.578785 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57937a-230a-4ec1-b6a6-0b312a8317cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:31:30.598413 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-ae57937a-230a-4ec1-b6a6-0b312a8317cb None None] MFA Rules not processed for user `a71f15d0547a4d78b6d540e5eb5b2fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:31:30.599618 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-ae57937a-230a-4ec1-b6a6-0b312a8317cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:31:30.607644 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1440/2891] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:31:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:31:40.592510 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=3 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:11.680402 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2833b2-eb12-494b-bcc5-213c43421da4 service placement] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:33:11.690308 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2833b2-eb12-494b-bcc5-213c43421da4 service placement] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:33:11.693931 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c202d-6c07-43ce-9f2d-12dd6e307819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:33:11.694518 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c202d-6c07-43ce-9f2d-12dd6e307819 service neutron] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:33:11.694793 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c202d-6c07-43ce-9f2d-12dd6e307819 service neutron] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:33:11.695065 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c202d-6c07-43ce-9f2d-12dd6e307819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:33:11.697033 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f16c202d-6c07-43ce-9f2d-12dd6e307819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:33:11.707092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c202d-6c07-43ce-9f2d-12dd6e307819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:33:11.708499 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c202d-6c07-43ce-9f2d-12dd6e307819 service neutron] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:33:11.716421 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1452/2892] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:33:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:33:21.712902 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=6 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:31.847172 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f631b-c500-4cf3-88ef-c144323abefe None None] REQUEST_METHOD: `POST` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:33:31.847837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f631b-c500-4cf3-88ef-c144323abefe None None] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:33:31.847837 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f631b-c500-4cf3-88ef-c144323abefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:33:31.867405 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.auth.core [None req-a12f631b-c500-4cf3-88ef-c144323abefe None None] MFA Rules not processed for user `a71f15d0547a4d78b6d540e5eb5b2fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81930) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:33:31.868563 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-a12f631b-c500-4cf3-88ef-c144323abefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:33:31.875705 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1441/2893] 10.222.0.30 () {58 vars in 958 bytes} [Wed Aug 24 10:33:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:33:31.917906 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c202d-6c07-43ce-9f2d-12dd6e307819 service neutron] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:33:31.919348 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-f16c202d-6c07-43ce-9f2d-12dd6e307819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:33:31.979862 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c202d-6c07-43ce-9f2d-12dd6e307819 service neutron] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:33:31.983469 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b4955-cdd8-4def-a8ff-9fef1179601b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c71783d1327a47149a74414218dd7219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:33:31.984070 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b4955-cdd8-4def-a8ff-9fef1179601b service cinder] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:33:31.984352 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b4955-cdd8-4def-a8ff-9fef1179601b service cinder] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:33:31.985533 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b4955-cdd8-4def-a8ff-9fef1179601b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:33:31.987458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-e89b4955-cdd8-4def-a8ff-9fef1179601b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:33:32.001992 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b4955-cdd8-4def-a8ff-9fef1179601b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:33:32.003427 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b4955-cdd8-4def-a8ff-9fef1179601b service cinder] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:33:32.014772 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1453/2894] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:33:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:33:33.259267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f631b-c500-4cf3-88ef-c144323abefe None None] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:33:33.271801 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f631b-c500-4cf3-88ef-c144323abefe None None] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:33:33.277208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930558a-0ca8-48d0-a4bc-82b80c86a90d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:33:33.277208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930558a-0ca8-48d0-a4bc-82b80c86a90d service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:33:33.277208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930558a-0ca8-48d0-a4bc-82b80c86a90d service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:33:33.277208 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930558a-0ca8-48d0-a4bc-82b80c86a90d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:33:33.278987 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c930558a-0ca8-48d0-a4bc-82b80c86a90d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:33:33.294443 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930558a-0ca8-48d0-a4bc-82b80c86a90d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:33:33.295893 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930558a-0ca8-48d0-a4bc-82b80c86a90d service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:33:33.311966 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1442/2895] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:33:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:33:37.358421 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b4955-cdd8-4def-a8ff-9fef1179601b service cinder] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:33:37.371343 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b4955-cdd8-4def-a8ff-9fef1179601b service cinder] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:33:37.379504 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a049767-63a2-4e52-942c-863ae2f83eb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:33:37.380180 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a049767-63a2-4e52-942c-863ae2f83eb1 service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:33:37.380458 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a049767-63a2-4e52-942c-863ae2f83eb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:33:37.381092 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a049767-63a2-4e52-942c-863ae2f83eb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:33:37.382999 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-1a049767-63a2-4e52-942c-863ae2f83eb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:33:37.397835 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a049767-63a2-4e52-942c-863ae2f83eb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:33:37.399622 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a049767-63a2-4e52-942c-863ae2f83eb1 service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:33:37.412243 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1454/2896] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:33:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:33:43.303342 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=9 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:47.406496 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=32 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:51.126068 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930558a-0ca8-48d0-a4bc-82b80c86a90d service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:33:51.128124 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c930558a-0ca8-48d0-a4bc-82b80c86a90d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:33:51.141922 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930558a-0ca8-48d0-a4bc-82b80c86a90d service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:33:51.147362 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b251a1-2aa0-403f-9824-b368bf02fefd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:33:51.147957 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b251a1-2aa0-403f-9824-b368bf02fefd service nova] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:33:51.148240 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b251a1-2aa0-403f-9824-b368bf02fefd service nova] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:33:51.148519 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b251a1-2aa0-403f-9824-b368bf02fefd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:33:51.156356 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b251a1-2aa0-403f-9824-b368bf02fefd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:33:51.158534 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b251a1-2aa0-403f-9824-b368bf02fefd service nova] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:33:51.167454 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1443/2897] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:33:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:33:53.492013 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a049767-63a2-4e52-942c-863ae2f83eb1 service nova] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:33:53.492613 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a049767-63a2-4e52-942c-863ae2f83eb1 service nova] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:33:53.498202 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072599a7-c6a4-454e-a366-c9c32e1d819f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cd6d2291f934fbdb25c410c2c3abcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:33:53.498806 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072599a7-c6a4-454e-a366-c9c32e1d819f service nova] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:33:53.499085 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072599a7-c6a4-454e-a366-c9c32e1d819f service nova] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:33:53.499360 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072599a7-c6a4-454e-a366-c9c32e1d819f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:33:53.501108 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-072599a7-c6a4-454e-a366-c9c32e1d819f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:33:53.514112 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072599a7-c6a4-454e-a366-c9c32e1d819f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:33:53.515698 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072599a7-c6a4-454e-a366-c9c32e1d819f service nova] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:33:53.527463 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1455/2898] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:33:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:34:01.163331 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=6 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:03.522660 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=6 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:36.318237 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b251a1-2aa0-403f-9824-b368bf02fefd service nova] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:35:36.355975 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b251a1-2aa0-403f-9824-b368bf02fefd service nova] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:35:36.359634 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0a198-4674-48a1-903d-f5908b81bd4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9b6525009b41288dd918437acee92a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:35:36.360232 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0a198-4674-48a1-903d-f5908b81bd4c service neutron] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:35:36.360514 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0a198-4674-48a1-903d-f5908b81bd4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:35:36.360788 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0a198-4674-48a1-903d-f5908b81bd4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:35:36.450381 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0a198-4674-48a1-903d-f5908b81bd4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:35:36.451743 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0a198-4674-48a1-903d-f5908b81bd4c service neutron] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:35:36.470838 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1444/2899] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:35:40.108309 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072599a7-c6a4-454e-a366-c9c32e1d819f service nova] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:35:40.157925 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072599a7-c6a4-454e-a366-c9c32e1d819f service nova] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:35:40.163643 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487c2f3-b42c-4a22-885e-5f1e2fbbdc67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:35:40.164251 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487c2f3-b42c-4a22-885e-5f1e2fbbdc67 service placement] REQUEST_METHOD: `GET` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:35:40.164516 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487c2f3-b42c-4a22-885e-5f1e2fbbdc67 service placement] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:35:40.164788 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487c2f3-b42c-4a22-885e-5f1e2fbbdc67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:35:40.166568 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-2487c2f3-b42c-4a22-885e-5f1e2fbbdc67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:35:40.187323 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487c2f3-b42c-4a22-885e-5f1e2fbbdc67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:35:40.188970 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487c2f3-b42c-4a22-885e-5f1e2fbbdc67 service placement] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:35:40.198953 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1456/2900] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:35:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:35:43.749675 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0a198-4674-48a1-903d-f5908b81bd4c service neutron] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:35:43.763171 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0a198-4674-48a1-903d-f5908b81bd4c service neutron] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:35:43.764539 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:35:43.765219 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:35:43.765500 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:35:43.765777 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] PATH_INFO: `/v3/users/a71f15d0547a4d78b6d540e5eb5b2fbc` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:35:43.766907 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a71f15d0547a4d78b6d540e5eb5b2fbc)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:35:43.769397 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:35:44.044278 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a71f15d0547a4d78b6d540e5eb5b2fbc'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:44.045046 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a71f15d0547a4d78b6d540e5eb5b2fbc'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:44.052257 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a71f15d0547a4d78b6d540e5eb5b2fbc'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:44.063778 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a71f15d0547a4d78b6d540e5eb5b2fbc'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:44.171594 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a71f15d0547a4d78b6d540e5eb5b2fbc'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:44.177862 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1445/2901] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:35:43 2022] DELETE /identity/v3/users/a71f15d0547a4d78b6d540e5eb5b2fbc => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:35:44.424957 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487c2f3-b42c-4a22-885e-5f1e2fbbdc67 service placement] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:35:44.439326 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487c2f3-b42c-4a22-885e-5f1e2fbbdc67 service placement] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:35:44.498734 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:35:44.499368 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:35:44.499648 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:35:44.499922 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] PATH_INFO: `/v3/projects/8013ed9ae5ed46b9bc656f193c43e118` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:35:44.501054 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8013ed9ae5ed46b9bc656f193c43e118)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:35:44.502989 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:35:44.611898 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8013ed9ae5ed46b9bc656f193c43e118', 'request_id': 'req-81247e66-9b8d-40f5-9bab-f930e215c7c0'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:44.618266 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8013ed9ae5ed46b9bc656f193c43e118', 'request_id': 'req-81247e66-9b8d-40f5-9bab-f930e215c7c0'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:44.624583 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8013ed9ae5ed46b9bc656f193c43e118', 'request_id': 'req-81247e66-9b8d-40f5-9bab-f930e215c7c0'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:44.739817 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] The token cache is being invalidate because project 8013ed9ae5ed46b9bc656f193c43e118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:35:44.740048 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:44.740666 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:44.742212 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1457/2902] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:35:44 2022] DELETE /identity/v3/projects/8013ed9ae5ed46b9bc656f193c43e118 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:35:44.746177 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:35:44.747611 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:35:44.756412 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:44.761729 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5114a-8ea4-4e0b-a408-1bc08f20756f admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:35:44.820227 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:35:44.820866 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:35:44.821173 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:35:44.821448 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] PATH_INFO: `/v3/users/b6639eb8aa1b4b7db0e559a9d155c9a3` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:35:44.822734 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6639eb8aa1b4b7db0e559a9d155c9a3)` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:35:44.824703 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:35:45.128616 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6639eb8aa1b4b7db0e559a9d155c9a3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:45.129968 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6639eb8aa1b4b7db0e559a9d155c9a3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:45.136652 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6639eb8aa1b4b7db0e559a9d155c9a3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:45.149484 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6639eb8aa1b4b7db0e559a9d155c9a3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:45.263387 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.notifications [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6639eb8aa1b4b7db0e559a9d155c9a3'} {{(pid=81930) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:45.270444 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1446/2903] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Aug 24 10:35:44 2022] DELETE /identity/v3/users/b6639eb8aa1b4b7db0e559a9d155c9a3 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:35:45.516726 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] Authenticating user token {{(pid=81931) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:35:45.518196 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.fernet_utils [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81931) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:35:45.530988 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81247e66-9b8d-40f5-9bab-f930e215c7c0 admin admin] Validating token access rules against request {{(pid=81931) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:35:45.556653 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:45.588259 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb4aa9488e54d0eafc940ddd57f345d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85765b8a176a42c8b2b1e092bbcf9c4f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81931) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:35:45.588908 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:35:45.589190 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] SCRIPT_NAME: `/identity` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:35:45.589468 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] PATH_INFO: `/v3/projects/64b205bd5ef941a4a38182871d4cdbde` {{(pid=81931) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:35:45.590558 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] RBAC: Authorizing `identity:delete_project(project_id=64b205bd5ef941a4a38182871d4cdbde)` {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:35:45.592440 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] RBAC: Authorization granted {{(pid=81931) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:35:45.744072 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64b205bd5ef941a4a38182871d4cdbde', 'request_id': 'req-65af9281-d8de-4afa-aed5-b37d6867329f'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:45.749515 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64b205bd5ef941a4a38182871d4cdbde', 'request_id': 'req-65af9281-d8de-4afa-aed5-b37d6867329f'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:45.755933 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64b205bd5ef941a4a38182871d4cdbde', 'request_id': 'req-65af9281-d8de-4afa-aed5-b37d6867329f'} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:45.822740 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] The token cache is being invalidate because project 64b205bd5ef941a4a38182871d4cdbde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81931) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:35:45.822984 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:45.823633 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG keystone.notifications [None req-65af9281-d8de-4afa-aed5-b37d6867329f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81931) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:35:45.825250 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: [pid: 81931|app: 0|req: 1458/2904] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Aug 24 10:35:45 2022] DELETE /identity/v3/projects/64b205bd5ef941a4a38182871d4cdbde => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:35:55.266449 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=52,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:55.819551 dv-cinder-854271-2-iscsi devstack@keystone.service[81931]: DEBUG dbcounter [-] [81931] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81931) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:13.968615 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] Authenticating user token {{(pid=81930) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:36:13.970106 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:36:14.061196 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6d42c-6473-4376-8eb5-94b468d4db1b admin admin] Validating token access rules against request {{(pid=81930) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:36:14.121267 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374deca-4b69-4673-af68-8769a51a2785 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1aa47a883604336a8ab90d324470477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e7b34d5e4a480691bed8235d4722c8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81930) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:36:14.121903 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374deca-4b69-4673-af68-8769a51a2785 service placement] REQUEST_METHOD: `GET` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:36:14.122185 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374deca-4b69-4673-af68-8769a51a2785 service placement] SCRIPT_NAME: `/identity` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:36:14.122460 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374deca-4b69-4673-af68-8769a51a2785 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81930) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:36:14.124222 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.fernet_utils [None req-2374deca-4b69-4673-af68-8769a51a2785 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81930) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:36:14.135329 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374deca-4b69-4673-af68-8769a51a2785 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:36:14.137060 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374deca-4b69-4673-af68-8769a51a2785 service placement] RBAC: Authorization granted {{(pid=81930) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:36:14.147080 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: [pid: 81930|app: 0|req: 1447/2905] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Aug 24 10:36:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:36:24.142629 dv-cinder-854271-2-iscsi devstack@keystone.service[81930]: DEBUG dbcounter [-] [81930] Writing DB stats keystone:SELECT=67 {{(pid=81930) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}